#pathom

generated UTC: 2023-02-13 19:07
latest data: https://clojurians-log.clojureverse.org/pathom/2023-02-11
messages: 11352
pro tips:
* Double click on text to filter by it. (doubleclick + cmd-f for extra points).
* Click on date to keep day visible regardless of filter.
* Click on time to keep hour visible regardless of filter.
#2018-10-2309:05mitchelkuijpersI am looking for some help I am currently fighting with joins in pathom, I am running pathom 2.2.0-beta14. And I am trying the following:
(defresolver `entity-autocomplete
  {::pc/output [{:entity/autocomplete
                 [::autocomplete/group-key
                  ::autocomplete/group-entity-type
                  {::autocomplete/items {:entity.type/company [:company/id :company/name :company/website :company/description]
                                         :entity.type/contact [:company/id :contact/name :contact/email :contact/description]}}]}]}
  (fn [{:keys [ast] :as env} _]
    {::pc/env (assoc env ::p/union-path :entity/type)
     :entity/autocomplete search-results}))
When I run the following test:
(is (= (test-parser env
             [{'(:entity/autocomplete {:filter-value "Av"})
               [::autocomplete/group-key
                ::autocomplete/group-entity-type
                ::autocomplete/items {:entity.type/contact [:contact/id :entity/type]
                                      :entity.type/company [:company/id :entity/type]}]}])

           [{::autocomplete/group-entity-type :entity.type/company
             ::autocomplete/group-key :atlas-crm.domain.autocomplete.group-key/search-results
             ::autocomplete/items [{:company/id 1
                                    :company/name "Avisi"
                                    :entity/type :entity.type/company}]}]))
I am getting very weird output which I don't expect I am getting a :entity.type/contact :com.wsscode.pathom.core/not-found below :entity/autocomplete which I don't expect
#2018-10-2309:05mitchelkuijpersThis diff shows it more clearly#2018-10-2311:19wilkerluciohello @mitchelkuijpers, long time šŸ™‚ what reader are using you using? the regular pc/reader?#2018-10-2311:54wilkerlucio@mitchelkuijpers I just noticed your query is missing the {} around the ::autocomplete/items join#2018-10-2311:55mitchelkuijpersOh goodness#2018-10-2311:55mitchelkuijpersThank you so much!#2018-10-2311:55wilkerlucioand the output should be a map insteadf of a vector, but I guess that will show up quickly there now, hehe#2018-10-2311:55mitchelkuijpers@wilkerlucio We are using pc/reader but I as peeking into parallel reader#2018-10-2311:57wilkerluciocool, if you are going from the serial parser you have some things to check during the migration#2018-10-2311:57wilkerlucioI'm gonna write more about it, but the main thing is sync vs async#2018-10-2311:57mitchelkuijpersNice this totally fixes it! We looked at it with 3 persons but no-one noticed šŸ˜›#2018-10-2311:58mitchelkuijpersCan the parallel reader do smarter batches? Because then it would be very interesting#2018-10-2311:58wilkerluciopeople migrating from async parser will have no work do to, but if you wrote plugins or you do recursive parser calls in your resolvers, then you need to change some things to support async because the parallel is async#2018-10-2311:59wilkerlucioit can do the same batch the serial does, is that one or you are thinking on something different?#2018-10-2311:59mitchelkuijpersAh we don't do recursive parser calls we just use connect#2018-10-2312:00mitchelkuijpersNot sure, we will do load-testing next week so we might try out both and report back if the parallel gives a big speedboost#2018-10-2312:00mitchelkuijpersI would guess so#2018-10-2312:01wilkerluciocool, another thing to watch out is the ::pc/env thing, that just doesn't work on parallel due to how it process the query, I'm replacing it with ::p/env that will work in a lower level and should work to port, but I have to verify if it's working in all expected cases#2018-10-2312:02mitchelkuijpersAh ok would be interested in that. But we acdtually added it to the env-plugin: ` ::p/union-path :entity/type `#2018-10-2312:03wilkerlucioat same time, for unions we have a different standard solution as well, that I think fits more easely in the connect way#2018-10-2312:03mitchelkuijpersAh ok would be interested in that. But we actually added it to the env-plugin:
::p/union-path :entity/type
#2018-10-2312:03wilkerluciooh, if you have that static, then all good#2018-10-2312:03mitchelkuijpersYes we do#2018-10-2312:03mitchelkuijpersAnd if we need more options I would make that a function#2018-10-2312:03wilkerlucioin previous versions if you didn't provide the ::p/union-path then it would raise an error on unions, but that changed#2018-10-2312:04mitchelkuijpersYes I noticed, very nice change#2018-10-2312:04wilkerlucionow the default is a fn that tries to find the union branch as a prop on the entity#2018-10-2312:04wilkerlucioeasier to code than explain XD#2018-10-2312:04wilkerlucio
(defn default-union-path [{:keys [query] :as env}]
  (let [e (entity env)]
    (if-let [path (some->> (keys query)
                           (filter #(contains? e %))
                           first)]
      path)))
#2018-10-2312:04mitchelkuijpersAh nice, that is fancy#2018-10-2312:05mitchelkuijpersI gotta to but thank you so much for spotting the derp#2018-10-2312:05wilkerluciono worries, glad to talk to you again, its been some time#2018-10-2714:06souenzzoCan I remove not-found entries from final result maps?#2018-10-2714:17souenzzoI tried to use p/elide-not-found as a reader and failed. Now using in the right way. But i think that I can do the same faster with #specter#2018-10-2719:20wilkerlucio@souenzzo yes, you should use the p/elide-not-found in combination with the p/post-process-plugin#2018-10-2719:20wilkerluciolooks like this: (p/post-process-parser-plugin (p/elide-not-found))#2018-10-2719:20wilkerlucioin the ::p/plugins list#2018-10-2721:24currentoorSo @wilkerlucio my parser looks like this, I added the ::pc/mutation-join-globals key
(def parser
  (p/parser
    {::p/mutate  server/server-mutate
     ::p/plugins [(p/env-wrap-plugin (fn [env]
                                       (assoc env ::pc/indexes @indexes)))
                  (p/env-plugin {::p/reader               [p/map-reader
                                                           pc/all-readers
                                                           (p/placeholder-reader ">")]
                                 ::p/placeholder-prefixes #{">"}
                                 ::pc/mutation-join-globals [::prim/tempids]
                                 ::pc/resolver-dispatch   resolver-fn})
                  p/request-cache-plugin
                  pp/profile-plugin
                  (preprocess-parser-plugin log-requests)
                  (preprocess-parser-plugin add-current-info)
                  (p/post-process-parser-plugin p/elide-not-found)]}))
#2018-10-2721:25currentoorand my mutation looks like this
(defmutation new-wash
  [{:keys [wash-id]}]
  (with [env] s.policy/existence)
  (action [{:keys [db conn current/firm current/user]}]
    (let [db-id (wash/create conn {:org-id  (:db/id firm)
                                               :user-id (:db/id user)})]
      {::prim/tempids {wash-id db-id}})))
#2018-10-2721:25currentoorbut the tempid remapping is still not happening#2018-10-2721:52wilkerlucio@currentoor I'm not using this in quite a time, so it's possible there is a bug, I'm currently updating the code in the docs, I can double check and get back to you if it's working properly there#2018-10-2721:52currentooroh ok, thanks#2018-10-2722:02wilkerlucio@currentoor I just tried running hte book code, it seems to work there, at least in terms of data return, can you check on inspect if the tempids key is been returned from the network call?#2018-10-2800:46currentoor@wilkerlucio yes the key is there#2018-10-2800:47currentoorhttps://lh3.googleusercontent.com/-1IihqiOrKVg/W9UHFaDZRYI/AAAAAAAAGHM/p9X0v1J7MWQuTzVqqSHb2vEKY609Kkv0ACLcBGAs/s0/Screen%2BShot%2B2018-10-27%2Bat%2B5.46.59%2BPM.png#2018-10-2800:48wilkerlucio@currentoor then you must check on the fulcro side of things, if the remote is getting the data the job is done in pathom side#2018-10-2800:49wilkerlucioI see your key on the map is another map, is that expected?#2018-10-2800:49wilkerluciofeels like it should have just the id there (not a map with id in it)#2018-10-2800:49currentooryeah that’s the weird part#2018-10-2800:50currentoorseems like it looses it’s tempid encoding#2018-10-2800:50currentoori believe tempids under the hood are encoded like that#2018-10-2800:51wilkerlucioyou should check your transit configuration, you need to support tempids there#2018-10-2800:51wilkerluciothe fulcro default one usually works, but you also have to check your server side#2018-10-2800:51currentooryeah weird stuff, well thanks for the help#2018-10-2800:51currentoorat least we know pathom is doing what its supposed to simple_smile#2018-10-2800:52wilkerlucioyup šŸ™‚#2018-10-2800:53currentoorbut one thing to note is this was working before we moved the parser to pathom#2018-10-2800:55wilkerlucioyou just changed the parser? you can try checking the types in an out before encoding, might tell something#2018-10-2800:56wilkerlucioI gotta go now, but let me know if you figure more out#2018-10-2802:18currentoorno worries#2018-10-2820:15currentooris there a way to make exception bubble up to the client?#2018-10-2820:17currentoori was relying on mutations throwing exceptions when something goes wrong#2018-10-2820:24currentoorfor example i do authorization check and throw an exception if permissions are not valid#2018-10-2820:25wilkerlucio@currentoor the error plugin is there to prevent that, you can use the flag ::p/fail-fast? true to force it bubble, but I suggest you also consider another approach, what I usually do is return a map with some special key like ::p/errors that tells me that request failed#2018-10-2820:25wilkerluciothis way I can handle it on the client like any other data#2018-10-2820:26wilkerluciobut if you really want to keep mutations blowing, you can just modify the error-handler-plugin, if you look at its definition this is what you will find:#2018-10-2820:26wilkerlucio
(def error-handler-plugin
  {::wrap-read   wrap-handle-exception
   ::wrap-parser wrap-parser-exception
   ::wrap-mutate wrap-mutate-handle-exception})
#2018-10-2820:26wilkerlucioso you can dissoc the ::wrap-mutate to remove the error catcher from mutations only#2018-10-2820:27wilkerluciolike: ::p/plugins [(dissoc p/error-handler-plugin ::p/wrap-mutate)]#2018-10-2820:31currentoorthanks @wilkerlucio, i’ll try the other approach you recommend#2018-10-2820:45currentoorSo using the default error handler plugin, now i’m getting the response back as {:com.wsscode.pathom.core/reader-error "class clojure.lang.ExceptionInfo: Unauthorized Request..."}#2018-10-2820:46currentoorwhich is nicer, but how do I change the key that the error is on?#2018-10-2820:47currentoori know there is ::p/process-error but that seems like it’s only for changing the value of the error, not the key#2018-10-2820:49currentooroh wait never mind, it actually let’s you specify the whole map#2018-10-2820:49currentoorthat might work#2018-10-2820:52wilkerlucioyeha, process-error is the way to go to give a custom handling šŸ˜‰#2018-10-2821:02currentoorso one other thing i want is to change the status of the response to show an error#2018-10-2821:02currentoorthat happens one level up from where process-error is being called#2018-10-2821:03currentoorany parser magic to do that?#2018-10-2821:04currentoorbecause if i put :status 401 in the result of process-error it puts it in the body of the response#2018-10-2821:05wilkerlucioyeah, that's by design to separate the error data from the response data, there is no way to change the up map, but in your UI you can instead render the ::p/reader-error map as the error data#2018-10-2821:06wilkerlucioor there is something that makes this pulling a problem in your scenario?#2018-10-2821:06currentoorwell fulcro-websockets has a global-error-callback#2018-10-2821:07currentoori was using that to log errors and redirect to login (if the status was a 401) or show some other error message#2018-10-2821:08currentoorthe UI render ::p/reader-error works only for errors related to what is being rendered#2018-10-2821:09currentoorbut if an auth token is expired i’d like to show a top level error then redirect to re-login#2018-10-2821:09currentoorunless i’m mistaken about something#2018-10-2821:09wilkerlucioare you using the pessimist mutations from fulcro incubator?#2018-10-2821:09currentoorin some places yes#2018-10-2821:09currentoorbut not for everything#2018-10-2821:10wilkerluciobecause thats a good case for it, since it provides the error-action you can use that to handle any redirect concerns#2018-10-2821:11currentooryes it works for that, but i don’t always want to use pmutate!#2018-10-2821:11currentoorfor one it can only do one mutation not a sequence#2018-10-2821:13wilkerlucioI'm looking at the error handler code, I might be able to make it return the full error map, this would make it more extensible, I'm thinking that it actually doesn't even work to the fulcro incubator case now#2018-10-2821:13wilkerluciobecause it always use the pathom reader error key#2018-10-2821:14wilkerluciowe can do sending some env flag like ::p/wrap-error? false so you get the full map control without breaking the previous behavior#2018-10-2821:15wilkerlucio@currentoor you think that works?#2018-10-2821:16currentoorwhat if we had a different key instead? env flag like that seems finicky#2018-10-2821:16currentoor::p/process-error-response for when you want to augment the whole response?#2018-10-2821:16currentoorrather that just the error#2018-10-2821:20wilkerlucioI like the idea, not the name, maybe ::p/process-error2, I still feel like the whole response should be the one to start with#2018-10-2821:20wilkerluciomaybe ::p/process-error*#2018-10-2821:21currentoorWhat do you mean ā€œthe whole response should be the one to start withā€?#2018-10-2821:21wilkerlucioI mean to return the whole map instead of a nested key#2018-10-2821:22currentoorWhat about::p/process-response-error#2018-10-2821:22currentoor?#2018-10-2821:22currentoorThat seems like a more descriptive name #2018-10-2821:24currentoorOh sorry I mean process-response-on-error #2018-10-2821:26wilkerluciothat can work, can you open an issue please?#2018-10-2821:27wilkerluciooh, wait#2018-10-2821:27wilkerlucioI'm double checking now, I think if you do the process error you get the whole map control don't you?#2018-10-2821:28wilkerlucioso you mean in the case you want that error in the response root?#2018-10-2821:29currentooryes#2018-10-2821:29currentoorit looks like i can add a mutate-wrapper that does that right?#2018-10-2821:29wilkerluciothat's kind a leaky abstraction, what if you send 2 mutations and each get a different error status?#2018-10-2821:30currentoorwhy would each get a different error status?#2018-10-2821:30wilkerlucio2 different requests#2018-10-2821:32wilkerlucioso seems like what you want is some more in the area of a global flags for general errors, is that correct?#2018-10-2821:33currentoori’m not sure i understand what you mean#2018-10-2821:33currentoori just want errors to have a non 200 status#2018-10-2821:33currentoorpreferably i can set the status depending on the error#2018-10-2821:34wilkerlucioI'm trying to understand that you want to do global app handling of those errors instead of localized handling#2018-10-2821:35wilkerluciodo you have some function in your control that wraps the http calls your api makes?#2018-10-2821:36currentoorbasically i’m looking for error behavior of the base fulcro mutation parser, but now i’m starting to think that goes agains the design philosophy of pathom#2018-10-2821:36currentoorlike ring middleware? yes, but most of my mutations happen over websockets#2018-10-2821:38wilkerluciowell, it's all doable, it's make to be flexible, but I like to discourage because in principle the API tries to encourage a path with good fault tolerance, in the moment you make some information about a particular error leaks to the global response context, this kind hurts the principle, but at same time I always try to make it as flexible as possible, so I make it hard but still make it possible, so you can do it at your own risk šŸ˜‰#2018-10-2821:39wilkerlucioso, I was thinking you can write a simple plugin that wraps the parser and injects the http status in case something happens, you can control this state using an atom, here is an example:#2018-10-2821:40currentooris there a way to wrap the response, like ring middleware?#2018-10-2821:41wilkerlucio
(def http-global-status-plugin
  {::p/wrap-parser
   (fn [parser]
     (fn [env tx]
       (let [status-atom (atom nil)
             result (parser (assoc env ::http-status status-atom) tx)]
         (cond-> result
           @status-atom (assoc :http-status @status-atom)))))})
#2018-10-2821:41wilkerlucioyeah, wrap-parser wraps the whole thing#2018-10-2821:42wilkerlucioso in this case, you can then add some try-catch in a middlewere of your websocket call, and if it fails it can update this atom from the env so the status will be injected in the root#2018-10-2821:44currentoori see, so is it a bad idea if i just used p/post-process-parser-plugin to change the out going response on error#2018-10-2821:44currentooras a last plugin#2018-10-2821:45currentoorright now it gets as input
#:ucv.ui.mobile-pos.screens.camera{new-wash-&-unconfirmed-vehicle
                                   {:error "Unauthorized Request",
                                    :fulcro.incubator.pessimistic-mutations/mutation-errors
                                    "Unauthorized Request",
                                    :status 401,
                                    :body
                                    {:message
                                     "mutation ucv.ui.mobile-pos.screens.camera/new-wash-&-unconfirmed-vehicle unauthorized, s.policy/fail violated"}}}
#2018-10-2821:46wilkerlucioyou could also write this as a plugin on the client side#2018-10-2821:46wilkerlucioyou could wrap the network (pathom has helpers to make that easy) and just look at the mutations in the response (look for symbols at the response root)#2018-10-2821:47wilkerlucioand then you could pull the status from then, and at this point you have all the information (all the statuses) so you could take a decision to pick one in case of ambiguity#2018-10-2821:48wilkerluciohttps://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/fulcro/network.cljs#L122-L147#2018-10-2821:49currentooryeah that could work too#2018-10-2821:49currentoorthanks#2018-10-2821:51wilkerlucioyou'r welcome#2018-10-2917:41souenzzoHere you say that the "context" map should contain ::p.http/url Then call request that requires :url and :method. Is it a bug? Should it unqualify or should it req-un ? https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/diplomat/http/clj_http.clj#L9#2018-10-2917:46wilkerlucio@souenzzo the calls to request must use namespaced keywrods from the diplomat/http.cljc namespace#2018-10-2917:46wilkerlucioin case of the cljhttp this code (not written) would convert the namespaced keys to the plain ones that clj-http accepts#2018-10-2917:47wilkerlucio(the current version is just not ready, I started written and just stopped, but PR's are very welcome if you like to get that working :))#2018-10-2917:47wilkerluciothe fetch version is working if you like to get some inspiration: https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/diplomat/http/fetch.cljs#2018-10-2917:53souenzzoOk. I will try to PR šŸ™‚#2018-10-3000:32souenzzoHow to start workspaces in pathom project?#2018-10-3000:43wilkerlucio@souenzzo npx shadow-cljs watch workspaces#2018-10-3000:45souenzzoworkspaces did not help me. Should I start a simple http server to make an "echo" test on com.wsscode.pathom.diplomat.http.clj-http ?#2018-10-3000:46wilkerlucioyeah, you must on CLJ, I suggest a regular REPL for it#2018-10-3000:46wilkerluciodo you use cursive?#2018-10-3000:48souenzzoI already tested on repl. I'm talking about "unit" test. like com.wsscode.pathom.diplomat.http.fetch-test/test-request-async#2018-10-3000:49wilkerluciocool šŸ™‚#2018-10-3000:50wilkerluciomost of the library is tested using the standard clj test#2018-10-3000:50wilkerlucioyou can create a new file and write a simple one there at same folder, clj_http_test.clj#2018-10-3001:26souenzzosent#2018-10-3001:42wilkerluciothanks! merged, I'm currently in the middle of documenting some new things for the release of 2.2.0, should be out this week and will include the new driver#2018-10-3003:39souenzzoNow pathom have 2 "server" drivers šŸ˜„#2018-10-3003:43souenzzo@wilkerlucio I'm planning a new application and I will use pathom to the backend. There is any advantage to use pathom also on client? How do you use pathom ?#2018-10-3012:01wilkerlucio@souenzzo I just use one or the other (server or client), usually I recommend the client version when the user can't use the server one (generally people trying to use fulcro that don't want to invest in a server just to route things)#2018-10-3016:10wilkerlucioHello people, Pathom 2.2.0 is very close to be ready, I just released the 2.2.0-RC1, I've been a bit quite about it but this update brings a lot of new goodies to the library, the highlights are: * A guide to leverage the new features is provided in the book (no breaking changes from 2.1.0, you can update the library and upgrade the details when it's convenient for you): https://wilkerlucio.github.io/pathom/DevelopersGuide.html#_2_2_0_upgrade_guide * Parallel parser with automatic coordination using connect, I updated the docs prividing information about how it works, you can find at: https://wilkerlucio.github.io/pathom/DevelopersGuide.html#_code_pc_parallel_reader_code * Tracer feature that will replace the old profiler, the profiler before could only measure the times of reads, the tracer is more dynamic, event based tracing for the parser, it can annotate any sug-segments * New format to define resolvers/mutations that are pure data (just maps), this new method facilitates the use of shared resolvers/mutations, example libraries coming soon! * All the book examples were updated to use the new map format an encourage it, and there are new sections for parallel parser and other sections got updated * The book examples now use the Query Editor from Pathom Viz, so you can get auto-complete and tracing features as you try to book! (try it out, its pretty cool, try the parallel parser demo example: https://wilkerlucio.github.io/pathom/DevelopersGuide.html#_code_pc_parallel_reader_code) Full list of changes at: https://github.com/wilkerlucio/pathom/blob/master/CHANGELOG.md I'm releasing the RC1 here just to make sure the full release its stable, I recommend everyboedy to upgrade and please let me know if you find any issues, after a few days if nothing comes up I can release 2.2.0 and announce it broader, thanks!#2018-10-3016:13wilkerlucio@souenzzo your http driver is there too šŸ˜‰#2018-11-0210:10levitanong@wilkerlucio When using (p/post-process-parser-plugin p/elide-not-found), I get PathomRemote error: Error: No protocol method IWithMeta.-with-meta defined for type cljs.core.async.impl.channels/ManyToManyChannel: [object Object] when dealing with async/parallel parsers#2018-11-0210:33levitanongSo far I’ve worked around it with extend-type, and just making -with-meta a no-op, but I hope that doesn’t break anything.#2018-11-0211:35wilkerlucio@levitanong can you try again with 2.2.0-RC2 please?#2018-11-0211:36wilkerlucioI just realized the post-process-plugin wasn't handling async, and given the parallel parser is async then this problem happens#2018-11-0211:36wilkerlucioon RC2 it should proper handles it so you don't get channel read errors#2018-11-0211:43levitanong@wilkerlucio awesome! will check it when I can. Currently away from my machine. #2018-11-0216:13levitanong@wilkerlucio works! thanks man!#2018-11-0301:57souenzzoUsing the new com.wsscode.pathom.diplomat.http with lacinia. It's way easier to write tests. https://github.com/souenzzo/my-next-stack/blob/master/test/server/graphql_test.clj#2018-11-0313:34wilkerlucioare you writing a lacinia client that calls into pathom?#2018-11-0314:34souenzzoNo, just using diplomat.http inside lacinia. Lacinia also have a "context" that is passed to "resolves"...#2018-11-0307:06levitanong@wilkerlucio It appears ::prim/tempids only works with p/mutation-join-globals, but not when explicitly stated as an output of a remote mutation. The documentation suggests that both should work.#2018-11-0312:44wilkerlucio@levitanong both should work, did you tried adding it to the mutation join query?#2018-11-0314:57levitanong@wilkerlucio ah, I did not. I only placed it in ::pc/output.#2018-11-0314:57wilkerlucioyeah, the point is that your clients have to request it in the mutation join#2018-11-0314:58wilkerluciothe global thign is there so it's automatically added to the mutation join request, otherwise each client call would have to do it#2018-11-0314:58wilkerluciolike: [{(call-operation {}) [::fulcro/tempids]}]#2018-11-0314:58levitanongor, presumably in returning#2018-11-0314:59wilkerlucioyeah, returning will add to the mutation join, its equivalent#2018-11-0314:59wilkerluciobut then you component query will have to mention it šŸ˜‰#2018-11-0315:00levitanongindeed#2018-11-0315:00levitanonggood thing you made mutation-join-globals šŸ˜›#2018-11-0315:00levitanongso i’m curious now about the role of ::pc/output. seems a bit redundant#2018-11-0315:01wilkerlucioright now it doesn nothing for mutations, but it will in the future#2018-11-0315:01wilkerluciolike when you do a query in pathom viz, and I can auto-complete the attributes because I know the context#2018-11-0315:01wilkerluciothe idea for ::pc/output in mutations is so I know whats the expected context (data available) on the output, then I can auto-complete based on that#2018-11-0315:01wilkerluciojust not implemented yet#2018-11-0315:02wilkerluciosame thing for ::pc/params#2018-11-0315:02wilkerluciomakes sense?#2018-11-0315:02levitanongoh okay!#2018-11-0315:03levitanongbut can’t that be inferred from the mutation AST?#2018-11-0315:08wilkerluciono, this is about the output right#2018-11-0315:08wilkerluciowhat uses asks vs what is provided#2018-11-0315:09wilkerluciothe output is the output expected from the mutation (same as resolver)#2018-11-0315:09wilkerluciothe mutation is a generic function, it can return anything#2018-11-0315:09wilkerlucioso the output is some annotation so the engine can know what to expect from without without having to call it#2018-11-0315:10wilkerlucioalso the user can ask for things that are not directly in the output#2018-11-0315:10wilkerlucioin the mutations join guide for example: https://wilkerlucio.github.io/pathom/DevelopersGuide.html#_mutation_joins#2018-11-0315:10wilkerluciothe output is just :user/id, but the user can ask for :user/name, because the system has resolvers that knows how to do this transition#2018-11-0315:11wilkerlucioso user request != mutation output, makes sense?#2018-11-0315:12levitanongah yes, because you could have several different mutation calls joined to different queries#2018-11-0315:13levitanongso the output ā€œqueryā€ needs to be independent?#2018-11-0315:15wilkerlucioyes, correct#2018-11-0315:17levitanonggreat! i may have another bug for you#2018-11-0315:18levitanongmy remote data doesn’t seem to be getting normalized properly. It’s alright if the data structure is 2 levels deep, but at the third level, i’m getting lots of empty maps#2018-11-0315:19levitanongwhen i run prim/tree->db with the same query i’m using for both the ::pc/output and returning, it’s showing what i expect#2018-11-0315:19levitanongin other words, what gets merged into my app db is different from prim/tree->db#2018-11-0315:21levitanongam i misunderstanding how the parser works?#2018-11-0315:26wilkerlucionormalisation is really a pure fulcro thing, there is nothing pathom can do to interfeer with it#2018-11-0315:26wilkerluciothe shape of the data in the return looks correct? if so you must check if the component you are using to trigger the load has the correct query map (using components with idents)#2018-11-0315:33levitanongi’m pretty sure it’s correct because i can run prim/tree->db properly.#2018-11-0315:34levitanongPathom does remove keys that aren’t in the ::pc/output vector though, right?#2018-11-0315:34levitanongor rather, it attempts to massage the data such that it matches the ::pc/output query shape#2018-11-0315:36wilkerluciono#2018-11-0315:36wilkerluciothe output is only about what the initial context is, what gets back is whatever you ask in your query#2018-11-0315:36wilkerlucioare you using fulcro inspect?#2018-11-0315:37wilkerluciowhat really matters is whatever is returning from your remote call#2018-11-0315:37wilkerlucioyou can see it with fulcro inspect#2018-11-0315:37wilkerluciolet me know the output, and what are you using in returning#2018-11-0315:37wilkerlucioso we can check whats going on#2018-11-0315:38wilkerlucio::pc/output is always in terms of "declaration", what this resource can do (be a mutation or resolver), your query output will always be about what you requested, and never anything more#2018-11-0315:41levitanongI do have fulcro inspect, but i’ve been looking at the data returned in
(p/post-process-parser-plugin
  (fn [input]
    (js/console.log "input" input)
    input))
#2018-11-0315:41levitanongI will check the remote section of fulcro inspect#2018-11-0315:42wilkerlucioits better to look there since that's what fulcro is receiving, so you are sure nothing is changing somewhere else#2018-11-0315:42wilkerlucioand can I see your returning? the declaration and the component its using#2018-11-0315:42wilkerlucioand the remote response if you can šŸ™‚#2018-11-0315:43levitanongOkay, hang on!#2018-11-0315:46wilkerlucioin the remote tab, you can click in the button "Send to query" to send a remote query to the Query tab#2018-11-0315:46wilkerlucioshould be easy to copy the result running it from there#2018-11-0315:47levitanongoh#2018-11-0315:47levitanongLOL#2018-11-0508:35levitanong@wilkerlucio is there any way to get the parsers to work with idents that don’t fit the classical pathom ident style? For example, I want to do a
(df/load reconciler [:foo/by-id 0] Foo {})
I am aware of open-ident-reader, but it doesn’t seem to do the trick. I’m trying to write my own fulcro-ident-reader, but i keep coming back to using prim/db->tree within that reader. Not sure if what i’m doing is sane šŸ˜›
#2018-11-0510:46wilkerlucioabout the ident, the thing is, the pathom design is made to remove the separation between what is data and what are entry points, the by-id concept to me just looks like you are giving id a second name, you can do it, I just dont see any gains, you add complexity by adding one unescessary name to your system, and adding effort to the engine to keep converting it#2018-11-0510:46wilkerlucioyou can create resolvers that convert by-id to id and vice versa, so they get to be the same#2018-11-0511:12wilkerlucio@levitanong can you tell me more about why you are using db->tree? and in which point? you should never need it#2018-11-0512:16levitanong@wilkerlucio I’m trying do (df/load reconciler [:foo/by-id 0] Foo {}). I initially built a reader that looked something like:
(defn fulcro-ident-reader
  [{:keys [ast state] :as env}]
  (let [{:keys [key component]} ast
        state-map               @state
        extra-content           (get-in ast [:params :pathom/context])
        entity                  (get-in state-map key)]
    (if (vector? key)
      (let [entity-with-extra (merge entity extra-content)]
        ;; Problem is, sometimes entity has this shape:
        ;; {:foo/id 0 :foo/bar [:bar/by-id 1]}
        ;; and I get an error in the value complaining about the value of`:foo/bar`
        ;; not being a sequence. I'm guessing it's trying to access the ident as a map
        (p/join (atom entity-with-extra) env))
      ::p/continue)))
#2018-11-0512:16wilkerluciowith connect you shouldn't need to write any new readers, also the context is also a bit of advanced feature, I wonder why you are hitting those already#2018-11-0512:17levitanongoh i just added the context thing because i looked at the source of open-ident-reader#2018-11-0512:18wilkerlucioin Connect its all about attributes, right, so when you do a query like [{[:foo/by-id 123] [:foo/bar]}]#2018-11-0512:18levitanongand i think it’s useful to separate the table name (like foo/by-id) from the identifying attribute (`foo/id`) because sometimes identifying a component is more complicated than just getting one property#2018-11-0512:18wilkerluciowhat you are doing is providing the data {:foo/by-id 123}#2018-11-0512:19wilkerlucionot really, identity is about a way to look the thing up, id is one option, but you can use any sort of thing, even new names#2018-11-0512:19wilkerluciomy problem is just having multiple names for the same thing (`id` and by-id), but you can still use those, but you have to write some resolvers to normalize it#2018-11-0512:19wilkerluciofor example#2018-11-0512:20wilkerlucio
(pc/defresolver foo-by-id->id [_ {:foo/keys [by-id]}]
  {::pc/input  #{:foo/by-id}
   ::pc/output [:foo/id]}
  {:foo/id by-id})

(pc/defresolver foo-id->by-id [_ {:foo/keys [id]}]
  {::pc/input  #{:foo/id}
   ::pc/output [:foo/by-id]}
  {:foo/by-id id})
#2018-11-0512:21wilkerlucioadding those you made :foo/id and :foo/by-id effectively equivalent#2018-11-0512:21levitanongooooh, that’s pretty cool#2018-11-0512:21levitanongdidn’t think you could do it that way#2018-11-0512:21wilkerluciobut I still suggest you avoid it, its just extra work for the parser šŸ˜›#2018-11-0512:22levitanongit would take a lot of tedious work to move away from the /by-id convention šŸ˜›#2018-11-0512:22levitanongi’ll let the parser do the extra work for now šŸ˜›#2018-11-0512:22wilkerluciook, hehe#2018-11-0512:22wilkerlucioin pathom is all about attributes#2018-11-0512:23wilkerlucioand you should be free to mess with then (translatre, join,separate...)#2018-11-0512:23wilkerluciobefore the parallel that thing above that I typed could result in bad loops, but the new implementation is smarter, doest fall there šŸ˜‰#2018-11-0512:25levitanongawesome!#2018-11-0512:25levitanongi suppose i’d have to keep pc/open-ident-reader for it to accept [:foo/by-id 0] since :foo/by-id wouldn’t be in the index#2018-11-0512:25wilkerluciocorrect#2018-11-0512:26wilkerlucioactually, no#2018-11-0512:26levitanongor… would the new defresolvers add those?#2018-11-0512:26levitanongyeah they would#2018-11-0512:26levitanongso it would just have to be pc/ident-reader#2018-11-0512:26wilkerlucioyeah, idents are generated when any resolver is added and it has a single input#2018-11-0512:26wilkerluciobut I personally prefer the open-ident-reader myself#2018-11-0512:26wilkerluciobecause there are cases where I want to load something and the ident just doesn't matter (singleton cases)#2018-11-0512:26wilkerlucioso the open-ident-reader still useful on those cases#2018-11-0512:27wilkerluciothe only reason to use the ident-reader is that if you wanna have extra ident readers in your system doing something else, so the ident-reader will forward the process when the ident is unknown#2018-11-0512:33levitanongoh, because it’ll just ::p/continue#2018-11-0512:35wilkerluciocorrect#2018-11-0512:36levitanongwould there be any case for having both?#2018-11-0512:36levitanongi assume that since it’s middleware-based, there is an order at which they are applied#2018-11-0512:36levitanongso you could have open-ident-reader as a catchall#2018-11-0512:36levitanongright?#2018-11-0512:36wilkerlucioyeah, that's possible#2018-11-0512:37wilkerlucioanother thing to keep in mind is about each reader performance#2018-11-0512:38wilkerluciomap-reader is quite fast, so it goes first (also because of caching and entity things), so always remember that to reach some reader far away, the ones before it must run, they are usually fast enough and good readers should short-circuit as soon as possible#2018-11-0512:39wilkerluciothis is more a thing to keep in mind if you are writing readers, but these days connect handles the great majority of things, so custom readers should not be so common#2018-11-0512:39levitanonggot it#2018-11-1309:46mitchelkuijpersHi all, I am switching over the the parallel parser of pathom but it seems it broke my union queries. Do I have to change something for them to work?#2018-11-1310:00mitchelkuijpersIf I remove my ::pc/union-path it starts working#2018-11-1310:08mitchelkuijpersI used to have {::pc/union-path :union-key} this does not work with the parallel parser changing it to:
::p/union-path (fn [{:keys [query] :as env}]
                               (let [e (p/entity env)]
                                 (:union-key e)))
Fixes it
#2018-11-1313:48wilkerlucio@mitchelkuijpers thanks for the report, looking at the code I can see the problem, I'll open a issue and deal with that soon#2018-11-1313:49wilkerluciohttps://github.com/wilkerlucio/pathom/issues/62#2018-11-1507:48maxt@wilkerlucio Hi! Can I get query params in a resolver somehow? For queries like ['(:find-events {:before "2018-01-01"})]#2018-11-1507:55wilkerluciohello, you can get it from the ast, like: (-> env :ast :params)#2018-11-1507:56maxtHm, I didn't see it there, I'll have another look, thanks#2018-11-1507:57wilkerlucioI think that's a missing topic in the book, but a good one to add#2018-11-1507:58wilkerluciothere is some info here: https://wilkerlucio.github.io/pathom/#Readers but quite outdated, params should a have a section somewhere else#2018-11-1508:06maxtIt might be that I confuse the connect api with the other. I'm trying to use defresolver, and the :ast doesn't seem to include params. Do I need a speical reader for that?#2018-11-1508:11maxtSorry, if I reduce my test-case I do get the :params in the :ast. I'll have another look at my code. Thank you for your reply!#2018-11-2418:57currentoorhow do i invoke the parser manually?#2018-11-2419:49wilkerlucio@U09FEH8GN what you mean by invoking the parser manually? the case you describe looks like a simple mutation join#2018-11-2420:02currentoorHow do I invoke the parser on a mutation literal outside of Fulcro #2018-11-2420:02currentoor?#2018-11-2504:45wilkerluciothe parser is a function, you can call it directly#2018-11-2504:45wilkerlucioif you are using on the client it will return a channel#2018-11-2504:45wilkerluciobut I'm not following what you need#2018-11-2418:57currentoorfor example i have a mutation
[{(ucv.ui.mobile-pos.screens.camera/new-wash-&-unconfirmed-vehicle
   {:file-id #uuid "a4028c1a-efdc-48f5-88aa-c0cf1b3015d3",
    :wash {:wash/id #uuid "53988eca-da63-424c-af8c-f1c67f4c8b26"}})
  [*]}]
#2018-11-2418:58currentoorbasically i want to connect the pathom parser to my file-upload remote so clients can send post-upload-mutation/reads along with file uploads#2018-11-2419:51wilkerlucio#2018-11-2621:11mitchelkuijpersAwesome release, the new batch-resolver helper is pretty nifty#2018-11-2622:29wilkerlucio@U060GQK8U glad you enjoyed! there is a cool trick you can use with the new resolver syntax in case you get to it, if you want to use the nice macro syntax and make a batch resolver, you can write it like this:
(pc/defresolver batched-resolver [env items]
  {::pc/input     #{:id}
   ::pc/output    [:id :name]
   ::pc/transform pc/transform-batch-resolver}
  (mapv #(get db (:id %)) items))
#2018-11-2709:18mitchelkuijpersAwesome, just changed it, to that. Like that waayyy better#2018-11-2709:29mitchelkuijpersAlso found some cases where we can us it ourselves, feels a bit like middleware.. very powerful#2018-11-2711:08wilkerluciofunction composition rox šŸ™‚#2018-11-2620:27souenzzoHow to query a thing with params in pathom? Example [{:app/todo [:todo/done?]}] ;; => {:app/todo [{:todo/done? true} {:todo/done? false}]} [{(:app/todo {:done? false}) [:todo/done?]}] ;; => {:app/todo [{:todo/done? false}]}#2018-11-2621:36wilkerlucioyou can pull the params from the ast#2018-11-2621:36wilkerlucio(-> env :ast :params)#2018-11-2701:55souenzzo@wilkerlucio
23:52:51.709 ERROR formiguinhas-service.parser - handler #error {
 :cause Call to #'com.wsscode.pathom.connect/pick-resolver did not conform to spec.
when I try [{(:app/todo {:done? false}) ...}] same with [({:app/todo ...} {:done? false})]
#2018-11-2702:08souenzzoIt's something with [:edn-query-language.core/join-query :edn-query-language.core/query :edn-query-language.core/query-expr :edn-query-language.core/property]#2018-11-2713:16souenzzoUsing (stest/unstrument 'com.wsscode.pathom.connect/pick-resolver) for now šŸ™‚#2018-11-2713:44wilkerluciothanks man, I'm in a bit rush because of the conj this week, but if you can file an issue I can take a better look next week#2018-11-2715:50souenzzoI will try to solve at night#2018-11-2720:19souenzzothere some thing really odd in pahtom instrumentation
(.toString xx)
Evaluation error (StackOverflowError) at java.util.regex.Pattern$Branch.match (Pattern.java:4736).
null
(.getMessage xx)
=> "Call to #'com.wsscode.pathom.diplomat.http/request did not conform to spec."
#2018-11-2720:34souenzzowhen we do (http/request (assoc ctx :url "abc123")) with (stest/instrument), it check all keys in ctx map and throws a really odd exception. It may be a clojure-spec issue too.#2018-11-2720:37souenzzoI'm using
(stest/unstrument '[com.wsscode.pathom.connect/pick-resolver
                    com.wsscode.pathom.connect/register
                    com.wsscode.pathom.connect/add
                    com.wsscode.pathom.diplomat.http/request])
#2018-11-2720:56wilkerluciothe :url needs to be namespaced to be valid to request#2018-11-2721:14souenzzoI know. it just a example šŸ™‚#2018-11-3021:55wilkerluciomy talk was released :) https://youtu.be/yyVKf2U8YVg#2018-12-0110:11thheller@wilkerlucio nice talk. thanks for finally giving the query language a proper name šŸ™‚#2018-12-0112:55wilkerluciothanks šŸ™‚#2018-12-0114:30myguidingstarthe talk is really fascinating @wilkerlucio#2018-12-0121:53wilkerluciothanks dude šŸ™‚#2018-12-0219:34Alex HI might be completely missing something, but how would you do the flat sort of join with a to-one in fulcro? can you have multiple (prim/get-query) at the same level?#2018-12-0219:36Alex Hwith the usual example of a cow having one farmer, and if I'd normally have pathom connect resolvers that return something to the extent of {:cow/id 1 :cow/name "Elsa" :farmer/id 9 :farmer/name "Rob"} - and I'd want them normalized sensibly in fulcro - what would the :query look like?#2018-12-0314:12wilkerluciothat';s the placeholder nodes, you could query like this:
[:cow/name {:>/farmer [:farmer/name]}]
#2018-12-0314:12wilkerluciomore info: https://wilkerlucio.github.io/pathom/#_placeholders#2018-12-0314:13wilkerluciojust noticed the default namespace is outdated there, will update in a bit#2018-12-0223:33souenzzoPathom users do [{[:user/by-id 33] [:user/name :user/id]}] or [{[:user/id 33] [:user/name :user/id]}] ? Then on fulcro db it should be :user/by-id {33 {...}} or :user/id {33 {...}} ?#2018-12-0303:37tony.kayrecommend :user/id, since that ends up working better with pathom’s flatten/expand model of attributes#2018-12-0306:58danielstockton
(def db
  (atom {:users [{:id 1}]}))

(pc/defresolver user-resolver [{:keys [database] :as env} {:keys [user/id]}]
  {::pc/input  #{:user/id}
   ::pc/output [:user/id]}
  (let [user (first (filter #(= (:id %) id)) (:users database))]
    {:user/id (:id user)}))

(def app-registry [user-resolver])

(def parser
  (p/parallel-parser
   {::p/env     {::p/reader               [p/map-reader
                                           pc/parallel-reader
                                           pc/open-ident-reader
                                           p/env-placeholder-reader]
                 ::p/placeholder-prefixes #{">"}}
    ::p/mutate  pc/mutate-async
    ::p/plugins [(pc/connect-plugin {::pc/register app-registry})
                 p/error-handler-plugin
                 p/request-cache-plugin
                 p/trace-plugin]}))
(<!! (parser {:database @db} [{[:user/id 3] [:user/id]}])) => {[:user/id 3] #:user{:id 3}}
I'm not sure why my parser keeps returning what I give it (user 'database' only has one user).
#2018-12-0314:07wilkerluciobecause you are trying to override a value that was already defined, it just read from the map#2018-12-0314:08wilkerlucioa resolver to rename itself is not a valid thing, because you would be given 2 different semantic properties under the same name (if that makes sense)#2018-12-0314:08wilkerluciowhen you provide the ident, the context is {:user/id 3}#2018-12-0314:09wilkerlucioso when you request :user/id it already in the context, its not gonna trigger any resolver, just return the value under the context#2018-12-0314:09wilkerluciomakes sense?#2018-12-0314:26danielstocktonYep, perfect, thanks!#2018-12-0314:52wilkerlucioUpdated docs for placeholder nodes https://wilkerlucio.github.io/pathom/#_placeholders#2018-12-0316:53aisamuThe > default is great, btw#2018-12-0401:50tony.kaySo, I’m seeing an issue with batch resolving mode…when I hit an endpoint that is batched, it doesn’t chain the the additional resolvers for that context#2018-12-0401:51tony.kayhm…no, that’s not it#2018-12-0402:02tony.kaynah, it must just be funky resolvers#2018-12-0616:40souenzzowhere should I put the access control restrictions? On every resolver? Inspect the query? Make a public-parser and a private-parser is a reasonable solution?#2018-12-0616:47wilkerluciousually on the resolver that's given access to the resource#2018-12-0616:47wilkerlucioif you think that every attribute might have a different access, breaking it on resolvers can give you a fine tune to allow or disallow something#2018-12-0616:48wilkerlucioyou can just throw an error on the resolver if the access check fails, makes sense?#2018-12-1022:48eoliphantwhat’s the best way to handle pagination with pathom?#2018-12-1112:37wilkerlucioI like to do it similar to how graphql handles connections, usually a query for a paginated resource looks like this:
[{:my-users-paginated
  [{:pagination/items
    [:name :email :address]}
   :pagination/cursor]}]
#2018-12-1112:38wilkerlucioso the items contains the actual items, while cursor is some map that has enough information to request the next page#2018-12-1112:38wilkerlucioso the next request can go like:#2018-12-1112:38wilkerlucio
[{(:my-users-paginated {:pagination/cursor ...})
  [{:pagination/items
    [:name :email :address]}
   :pagination/cursor]}]
#2018-12-1113:34eoliphantok I was wondering if the relay approach made sense šŸ™‚ But, of course, it’s way easier in pathom. I always liked the idea, but it ā€˜feels’ clunky to implement in graphql#2018-12-1220:53Alex HMight be a silly question, but it looks to me like queries with parameters essentially look the same as mutations. The only distinguishing factor, to some extent, is that the mutation uses a symbol instead of a keyword. Is that really all that sets them apart, syntax-wise?#2018-12-1221:10mitchelkuijpersYes you are completely right#2018-12-1221:10mitchelkuijpersNo silly question at all btw šŸ˜‰#2018-12-1309:43danielstocktonParameterized reads, rather than all queries. e.g. a parameterized join will have a map instead of a symbol.#2018-12-1316:18wilkerlucioyup, correct, lists with symbols as the first item are mutations, anything else is a parameterised query#2018-12-1419:52Alex Hthanks everyone for the clarifications!#2018-12-1614:54Alex Hhow do you get a hold of the parameters in a resolver? And where do those parameters need to be to be picked up by a given resolver? (e.g. around the join, around the join key, ...?)#2018-12-1614:57Alex Hdo they just get placed in the input (second argument?) alongside the actual inputs?#2018-12-1703:18wilkerlucio@alex340 you get params from the AST with (-> env :ast :params), there are 2 valid syntax, one with a list around the whole join or around the key, the AST of both are the same, I personally like the list around the key, I find it easier to read, but both should work fine#2018-12-1717:42souenzzo@wilkerlucio could have a (p/params ctx) or somethink like. There is one in my pathom-utils šŸ™‚ or maybe it will be in EQL repo#2018-12-1717:45wilkerluciomaybe in pathom since EQL since env is more a fulcro concern than a EQL concern, but really, its 2 key accesses away, not sure if worth having it#2018-12-1818:58Bjƶrn Ebbinghaus@wilkerlucio I have some trouble with the pathom remote for Fulcro. I alway get {:message "Mutation not found", :data {:mutation login}} when I make a mutation like so: [(login {...params...}})] I have used the template setup from the developers guide. What am I doing wrong here? I am getting crazy.
(pc/defmutation login [env {:keys [connection nickname password]}]
  {::pc/sym 'login
   ....})

(def my-app-registry [login])

(def parser
  (p/parallel-parser
    {::p/env {...}
     ::p/mutate pc/mutate-async
     ::p/plugin [(pc/connect-plugin {::pc/register my-app-registry})
                 ...]}))

#2018-12-1819:00wilkerlucio@mroerni the issue here I think is because you used a simple symbol to define your mutation, that means it has the fully qualified name whatever-ns-you-have/login instead of just login, I suggest you try namespacing it on the server (maybe user/login), of you use that on the definition then you can use directly#2018-12-1819:03Bjƶrn EbbinghausAlready did that. Same result. ::pc/sym 'dbas/login Is there a way to get all registered mutations?#2018-12-1819:04wilkerlucioyou can check on the index, there is ::index-mutations there that should have then all#2018-12-1819:09wilkerlucioI just realized there is no good full example of it, I'm writing one now so I can double check if its all good on that flow and provide an example#2018-12-1819:14Bjƶrn EbbinghausI am just starting with pathom. I don't fully understand where I get access to the index, yet.#2018-12-1819:18wilkerlucioto access it you need to provide your own, you can use (def index (atom {})) at some place, then provide to connect-plugin as ::pc/indexes index#2018-12-1819:19wilkerluciowhen the parser starts it will be mutated with the injected values#2018-12-1819:28Bjƶrn EbbinghausOh... Simple mistake.. It is: ::p/plugins not ::p/plugin šŸ˜“#2018-12-1819:34wilkerluciothe famous typos šŸ™‚#2018-12-1819:35Bjƶrn EbbinghausMaybe I should write a spec ...#2018-12-1819:39wilkerluciohere is the full simple demo so you can check other things as well šŸ˜‰ https://github.com/wilkerlucio/pathom/blob/master/workspaces/src/com/wsscode/pathom/workspaces/connect/simple_demo.cljs#2018-12-1820:46Bjƶrn EbbinghausThank you. šŸ™‚ My code is finally working like intended. After I gave up on my own remote, I realised that PathomRemote is already doing, what I intended to do.#2018-12-2020:36Alex HI've read through the documentation, and part of the code, and I just can't get an exception to actually be thrown when I throw it within a mutation. ::pp/fail-fast? doesn't do anything, the explicit error handler doesn't do anything. It seems to me like the code in pp/parser always does try/catch for a mutation, and so I always get a ::pp/error value instead of the exception being thrown.#2018-12-2020:38Alex HIn particular, this bit - the call (action) is what throws (down the line is the user-provided mutation of pathom connect), and that always get caught.
value (case type
                          :call
                          (do
                            (assert mutate "Parse mutation attempted but no :mutate function supplied")
                            (let [{:keys [action]} (mutate env key params)]
                              (if action
                                (try
                                  (action)
                                  (catch #?(:clj Throwable :cljs :default) e
                                    {::error e})))))
#2018-12-2020:45wilkerluciothere were changes with the parallel that made things more complected around error catching, I didn't tried but I guess the fail fast thing is broken, can you please file an issue for it?#2018-12-2020:45wilkerlucioand just wondering, for what purpose are you trying to get the extension to explode? its for debug it?#2018-12-2020:46Alex Hactually, I use the exceptions for validation, to be caught later in a ring handler, to format as a special exception response#2018-12-2020:47wilkerluciohumm, pathom is more focused on partial failure modes, its intended to never explode entirely#2018-12-2020:47wilkerlucioyou can have a request with multiple mutations and only some fails, pathom is designed to get something back to the user, so it never fails entirelyu#2018-12-2020:48wilkerluciomakes sense?#2018-12-2020:49Alex Hwell, yes, and I don't disagree. However, the straight-forward migration path for me is to have an overall error response#2018-12-2020:49Alex HI've raised https://github.com/wilkerlucio/pathom/issues/69 regarding the ::pp/fail-fast? behaviour.#2018-12-2020:51Alex HI'm only doing one mutation at a time, so the difference is mostly academic anyway. It's just easier for my client-side bits to recognize the error when it comes in that special format, rather than as a special result value in a normal mutation result.#2018-12-2020:52Alex Hit's mostly just showing an overall error message, effectively.#2018-12-2020:54wilkerluciocool, the reader part might need some more work, but I think to fail fast mutations will be an easy fix#2018-12-2020:54wilkerlucioI'll probably be able to get to it by the weekend, thanks for the report#2018-12-2021:01wilkerlucioah, I just tough a simple thing you can do to make it explode until then#2018-12-2021:01wilkerlucioyou can create a plugin around everything, look at mutation responses and trigger an error if you find any. makes sense?#2018-12-2021:01Alex Hyes, I was already looking into doing something like that#2018-12-2021:02Alex HOn a related note, it also seems that the error-handler-plugin is not called at all for errors in mutations#2018-12-2021:03Alex Hso if I was hoping to format it in some special way (not necessarily rethrowing), I don't see any way of doing that other than then going some other plugin that replaces the error values with (other) error values (or throws - as you suggested).#2018-12-2021:03wilkerlucioI think mutate-async is trapping it#2018-12-2021:03wilkerlucioyou can write a custom mutate-async, its a relative small fn#2018-12-2021:03wilkerlucioprobably the fix will be changing it#2018-12-2021:03Alex Hhm, I'm not using async parsers at all#2018-12-2021:03wilkerluciothis is the current impl:#2018-12-2021:03wilkerlucio
(defn mutate-async
  "Async mutate function to integrate connect mutations to pathom parser."
  [{::keys [indexes mutate-dispatch mutation-join-globals]
    :keys  [query]
    :or    {mutation-join-globals []}
    :as    env} sym' input]
  (if-let [{::keys [sym]} (get-in indexes [::index-mutations sym'])]
    (let [env (assoc-in env [:ast :key] sym)]
      {:action #(go-catch
                  (let [res (<?maybe (mutate-dispatch (assoc env ::source-mutation sym') input))]
                    (if query
                      (merge (select-keys res mutation-join-globals)
                             (<? (p/join (atom res) env)))
                      res)))})
    (throw (ex-info "Mutation not found" {:mutation sym'}))))
#2018-12-2021:04wilkerluciohumm, let me check the parallel parser#2018-12-2021:05wilkerlucioare you using parallel parser right? do your reads need async?#2018-12-2021:05Alex H
(def parser
  (p/parser {::p/env     {::p/reader [p/map-reader pc/reader2 pc/open-ident-reader p/env-placeholder-reader]
                          ::p/placeholder-prefixes #{">"}
                          ::p/process-error process-error
                          ::p/fail-fast? true}
             ::p/mutate  pc/mutate
             ::p/plugins [(pc/connect-plugin {::pc/register app-registry})
                          p/request-cache-plugin
                          p/trace-plugin
                          p/error-handler-plugin]}))

#2018-12-2021:06wilkerlucioweird, the old parser should work fine with fail-fast and error handler plugin#2018-12-2021:06Alex Hhow? that code segment I pasted in the bug report shows an unconditional try-catch around it, no?#2018-12-2021:07wilkerlucioyeah, I'm just wondering, because I was thinking of a problem somewhere else#2018-12-2021:10wilkerluciook, I'm gonna have to leave in a bit, sorry can't dig on this now, the regular parser and mutation parts could be interesting if you wanna investigate, by the weekend I'll have adequated time to look into#2018-12-2021:12wilkerluciooh, just read the code you posted on the issue, that makes sense, I gotta remember why I add that there#2018-12-2021:18Alex Hseems like only the parallel-parser does the right thing w.r.t. calling process-error#2019-12-3016:54wilkerlucio@U8Q71JK0W just released pathom 2.2.5, this removes the try catch on the parser level, please let me know if that works properly for you, thanks for the patience for this one šŸ™‚#2018-12-2815:03Alex KFirstly, thanks @wilkerlucio for Pathom. Looks like a superpower in the right hands. I'm having a bit of trouble figuring out how best to make it work with Datomic. I'm new to both (and Fulcro too), so I've been battling quite the learning curve. I still don't think I understand things as intended, so please forgive any silly questions (and also the message length). Q1) One thing that's confusing me is flattening out to-one relationships in the results map. For example, lets say I've got an entity "type" Person with an attribute :person/full-name, and another entity type Club with attributes :club/name, and :club/manager (which is a ref to a Person). I write a resolver that takes a Datomic :db/id for a Person entity and returns the attributes for that entity.
(pc/defresolver person-by-eid [{:keys [db] :as env} {:keys [db/id]}]
  {::pc/input #{:db/id}
   ::pc/output [:person/full-name]}
  (d/pull db [:person/full-name] id))
If I want to include the manager's :person/full-name in the same level of the Club resolver's result map, I'd have to remap it to a made up attribute e.g. :club/manager-full-name so that Pathom's indexes don't contain incorrect information (a :db/id for a Person returning :person/full-name, and also a :db/id for a Club returning :person/full-name, the latter being the wrong entity type).
(pc/defresolver club-by-eid [{:keys [db] :as env} {:keys [db/id]}]
  {::pc/input #{:db/id}
   ::pc/output [:club/name :person/full-name]}
  (let [club      (d/pull db [:club/name :club/manager] id)
        full-name (d/pull db [:person/full-name] (get-in club [:club/manager :db/id]))]
    (assoc club :club/manager-full-name full-name)))
As opposed to
(pc/defresolver club-by-eid [{:keys [db] :as env} {:keys [db/id]}]
  {::pc/input #{:db/id}
   ::pc/output [:club/name :person/full-name]}
  (let [club      (d/pull db [:club/name :club/manager] id)
        full-name (d/pull db [:person/full-name] (get-in club [:club/manager :db/id]))]
    (merge club full-name)))
I'm looking to inline a ton of attributes in places, which means I'd be inventing a ton of extra attribute names doing things this way. This isn't really a problem, but I feel like I'm misunderstanding Pathom, and taking some hacky routes to do things that would otherwise be straightforward. Am I going about this the right way? Q2) At the moment I've got resolvers taking a Datomic :db/id and pulling all the attributes for that entity type. I've also got UUIDs (e.g. :person/id) on many of the entities for external use, which I'm currently translating to a :db/id via a resolver, e.g.
(pc/defresolver person-by-id [{:keys [db] :as env} {:keys [person/id]}]
  {::pc/input #{:person/id}
   ::pc/output [:db/id]}
  (let [eid (first (first (d/q '[:find ?e
                                 :in $ ?id
                                 :where [?e :person/id ?id]]
                               db id)))]
    {:db/id eid}))
Is that the best way to be doing that? It would mean three or more resolvers where there are additional unique attributes that can be used as lookup refs, e.g. :person/email. Q3) For following Datomic refs backwards (underscore in front of the attribute in d/pull), I take it I'd just be writing another resolver that takes a :db/id for an entity, and returns the attributes for the entity on the other end of the ref (the same as following a ref in the usual direction)?
#2018-12-2817:12wilkerluciohello @noxdeleo, thanks for the kind feedback. About the questions, lets go over those: Q1/Q2: datomic is peculiar in the modeling because any entity can have any attribute, so you end up having to choose between: 1. make everything generic, just a single id an pull everything (I don't know anybody that tried that). 2. use specific ids for each "type", so you narrow the available properties given a specific id type, this is the most common and since you already have that for external apis I suggest you also use then on the pathom api so the derived attributes can be more predictable. If you do that way, then your case for person/club can look like this:
(pc/defresolver person-by-eid [{:keys [db] :as env} {:keys [db/id]}]
  {::pc/input #{:person/id}
   ::pc/output [:person/id :person/full-name :club/id]}
  (let [res (d/pull db [:person/id :person/full-name {:club/_manager [:club/id]}] id)]
    (-> res
        (assoc :club/id (get-in res [:club/_manager :club/id]))
        (dissoc :club/_manager))))

(pc/defresolver club-by-id [{:keys [db] :as env} {:keys [db/id]}]
  {::pc/input #{:club/id}
   ::pc/output [:club/id :club/name :person/id]}
  (let [res (d/pull db [:club/id :club/name {:club/manager [:person/id]}] id)]
    (-> res
        (assoc :club/id (get-in res [:club/manager :person/id]))
        (dissoc :club/manager))))
(there are some missing pieces, to convert the ids)
#2018-12-2817:12wilkerlucioso I think question 3 also goes in the same, does that makes sense?#2018-12-2817:22Alex KTotally, thanks! When I was struggling with this initially, I toyed with the idea of making a pluggable Datomic reader, similar to what you have for GraphQL. Sadly I'm time-limited with my current project, but once it's out the door, I'd be happy to give something like that a go if you think it would be of value? I'd dumped a few thoughts in a note for when I got around to it: * It could possibly resolve a query with a single d/pull if all nested attributes in (:query env) were in the Datomic schema. Guess that's only really an issue over the wire though? * Datomic schema already has ref cardinality, so maybe the reader could automatically flatten to-one joins into the results map. * Datomic schema lacks ref range, so the reader couldn't know what attributes the entity on the other end of a ref has without more information. Something along the lines of https://github.com/cognitect-labs/onto seems like it could help in this regard, while not reducing Datomic's schema flexibility to something like GraphQL types. I think point 3 covers that peculiarity in Datomic you mentioned.#2019-12-3017:29wilkerluciohello, thanks for the considerations, I think some bases to datomic would be nice, there are docs needed since so many people are trying to use this way#2019-12-3017:31wilkerlucioI think an advanced integration would be possible like we do on the GraphQL side, but that still needs some work to be fully featured. The auto flatten may not be a good idea because not every case is flattenable, hierarchies for example tend to not be possible, also if the entities share any attribute that might be a problem to flatten as well, so since its case sensitive I think it stays better as a manual thing#2019-12-3017:32wilkerluciowe need more exploration on this space for sure šŸ™‚#2019-01-0118:01Alex KCool. I watched a video yesterday from this year's Conj that looks like it might also make for a nice integration: https://github.com/luchiniatwork/hodur-engine. It's something I've been wanting to do with this latest project (I really like the idea of having a declarative domain model in one place and deriving other things from that). Plenty to think about and play with at any rate.#2019-01-0118:01Alex KI'm having a little issue with "optional" keys and computed keys in separate resolvers. I have entities in Datomic that sometimes never have an attribute, and entities that acquire one or more attributes later on when the user provides them. I've been querying for them anyway, using the elide-not-found plugin with a bit of conditional logic on the client, and everything was working ok. Then I added another resolver that takes an optional attribute as one of its keys. I was expecting it would only be called when this attribute existed in the parsing context (when the key's value is something other than ::p/not-found), but it's getting called anyway. The parser returns a load of reader errors because of "Insufficient resolver input". I saw in the docs a section about dependent attributes. Am I supposed to be using something like that in this situation? Interestingly, this also seems to be breaking something with the parallel parser/reader. Fulcro was complaining client-side about the Transit response (unexpected end of JSON or something like that), so I tapped the result of the parser to see what was going on. Some of the :com.wsscode.pathom.core/errors had a NullPointerException where there was exception information was for others. The number and position of these changed with each call. I replaced the parallel-parser with parser and parallel-reader with reader2, and the problem went away.#2019-01-0118:11wilkerluciohello, first lets talk about the transit error, the problem is encoding there, some error happened and was serialized as an error, you can fix that by supportign some default write handlers on the transit encoding, or changing how the error is processed, try adding to your environment: ::p/process-error p/error-str#2019-01-0118:11wilkerluciothis should convert the error to text, less data but easier to transmit#2019-01-0118:16Alex KHmm. Adding that is giving Exception in thread "async-dispatch-6" clojure.lang.ArityException: Wrong number of args (2) passed to: com.wsscode.pathom.core/error-str#2019-01-0118:17wilkerlucioabout the optional thing, that's a feature on the roadmap, currently all inputs are always required, the idea will be to mark parts as optional, currently a work around is to prefill the data with nils, for example, if you are loading one person entity with the fields [:person/name :person/age], you can have a resolver like:
clojure
(pc/defresolver person-resolver [_ _]
  {::pc/input  #{:person/id}
   ::pc/output [:person/name :person/age]}
  (let [person (pull-person ...)]
    (merge
      {:person/name nil :person/age nil}
      person)))
I opened the issue to track the optional attributes: https://github.com/wilkerlucio/pathom/issues/70
#2019-01-0118:18wilkerluciosorry. try as: ::p/process-error #(p/error-str %2)#2019-01-0118:27Alex KCool, that makes sense, thanks. Rich Hickey's latest Conj talk "Maybe Not" might be of interest. It seems he's changing spec with regards to optional attributes, breaking it out into "selectors" with a context. The gist is that attributes will be optional only at certain times, in certain contexts. So instead of required and optional keys in the specs, you use selectors to say what's needed at usage time. When I saw the slides for that, I immediately thought of EQL. Anyway, the correction to process-error worked, but unfortunately I'm still getting a broken Transit decode on the client. Unexpected end of JSON input.#2019-01-0118:28wilkerluciodid you kept your parser change? not sure if you did one extra bit that is the parallel parser is async, so it returns a chan, the regular is sync, is that part ok?#2019-01-0118:29wilkerlucioabout the selectors thing, in pathom case the input already is the selector part, so the options would be mark on it or have a second selection to mark part of the original selection as optional#2019-01-0118:33Alex KThat makes sense. I switched the parser back to parallel-parser and parallel-reader. I'm running the parser in Pedestal, so I'm using a blocking take with the parallel-parser, and took that out when I tried with parser2. So yeah, the blocking take is back in to get the parser response from the chan.#2019-01-0118:34wilkerlucioand still having the transit issue?#2019-01-0118:35wilkerluciohad you tried to read the transit output to check if it looks ok?#2019-01-0118:41Alex KYeah, the transit output is definitely malformed. It seems to stop abruptly somewhere in the middle of the :com.wsscode.pathom.core/errors section. That's why I wondered if those NullPointerExceptions were responsible.#2019-01-0118:47wilkerluciothats quite strange, did you checked the data before encoding to see if there is something strange? how are you seeing those NullPointerExceptions?#2019-01-0118:49Alex KI was using the new tap> functionality in Clojure 1.10 (with Puget as pretty printer) to dump the result of calling the parser. The NP exceptions were showing up there in the errors section of the parser output.#2019-01-0119:00wilkerluciocan you share that part of the output with me? I'm trying to understand from where those are coming from#2019-01-0119:01Alex KYeah, no problem. I'm just trying to see if the cutting-off point in transit coincides with a NullPointerException.#2019-01-0119:05Alex KHere's a gist of a run of the parser error section: https://gist.github.com/NoxDeleo/b9e9d671f2265f08e8db0d4882ca9670#2019-01-0119:08Alex KLooking at the transit, I can't find any correlation between stop point and those null pointers (it's really hard to read), however it is only malformed when the NP exceptions are in the output. Otherwise it comes through fine.#2019-01-0119:10Alex KIf it would help I can try to put together a minimal working example of the error?#2019-01-0119:28wilkerluciosure, if you can do that I can take a look and try to figure with you#2019-01-0120:01Alex KThis is frustrating. I can't get it to reproduce with a minimal example. I could zip up my project and send it your way if you'd like to try to chase this down, but I'd hate to waste more of your time if this is down to something weird that I've done.#2019-01-0121:46Alex KUh. Thought I'd caught a break, but not quite. I just implemented the workaround you suggested to nil the optional attributes, and now every error is giving back a NullPointerException every time. However applying that to my minimal example, that works as it should. I must have done something odd, and I'm not seeing it. I'll come back to it tomorrow with fresh eyes and hopefully figure this thing out. At least this is more information. Thanks for your help!#2019-01-0216:43Alex KHi again. So today I've been looking into this more, and I think there's something funky going on in Pathom's async error handling code when using the parallel-parser/reader. I noticed if I remove p/error-handler-plugin from the parser, things work ok. It's only with that plugin enabled that those NullPointerExceptions show up. I've been through the Pathom code trying to figure out how all that stuff works. I think I have the gist of what's happening when my resolver is being called, but the details are over my head. I don't know how you managed to write all that, haha. I also figured out why nilling the optional attribute meant a reader-error every time. I forgot to adjust the second resolver to deal with the nil input from the first, so that was causing a second error. D'oh! So unfortunately I still don't know how to make a minimal example that demonstrates the problem, so my example case is still my whole project. Should it help any, I have traces from an entity with the required inputs (resolver works), and another from the broken case where the required input is missing and the resolver fails.#2019-01-0213:58pauldThe pathom dev guide says that pathom-remote will use a pathom async parser. Can I use it with the parallel parser?#2019-01-0214:11pauldIt would be nice to have an example of fulcro using pathom. For instance, I don't know how to set the api endpoint and request-middleware without using fulcro's
fulcro-http-remote
#2019-01-0215:00pauldNevermind. I misunderstood. I can keep using fulcro-http-remote as long as I'm using pathom on the server.#2019-01-0215:38wilkerlucio@pauld correct, and about the async and parallel, yes, you can use, the parallel is an async parser#2019-01-0215:38pauldI just got it working!#2019-01-0215:38pauldJust had to use:
(def server-parser #(a/<!! (p/parser % %2)))
#2019-01-0215:39pauldI put that in place of the fulcro parser.#2019-01-0215:39pauldThanks.#2019-01-0221:56pauldIs there a trick to calling a mutation that's defined in a namespace other than the one that has the parser defined?#2019-01-0221:57pauldI get strange errors when I try to do this. On the browser console I get:
java.lang.Exception: Not supported: class java.lang.Class
#2019-01-0221:58pauldWhen I call it from my clojure repl I get:
#:cawala.api.mutations{delete-person
                       #:com.wsscode.pathom.core{:reader-error
                                                 java.lang.NullPointerException}}
#2019-01-0222:00pauldI'm not setting ::pc/sym. The app-registry is using
m/delete-person
#2019-01-0222:01pauldm is the alias for cawala.api.mutations namespace.#2019-01-0222:01pauldI'm returning nil from the mutation.#2019-01-0222:04pauldIt seems to work in my repl when called from the same namespace as my parser via:
`[(delete-person {:list-id 1 :person-id 1})]
#2019-01-0222:04pauldthat is if I move the definition of delete-person to the same namespace as parser#2019-01-0222:07pauld#2019-01-0222:36pauldI suspect my issue may be that I'm using the fulcro's cljs defmutation with pathom's clj defmutation.#2019-01-0222:43pauldOk I have a solution, not sure if it is the best way. I defined my mutation in the same namespace as parser but defined ::pc/sym to point to my mutations namespace. In that namespace I def'd my mutation by referring to the namespace containing parser.#2019-01-0312:26wilkerlucio@pauld yes, setting a custom ::pc/sym is the way to go, just trying to make the docs better, do you think we could more clear about it in the https://wilkerlucio.github.io/pathom/#_creating_mutations section?#2019-01-0416:01pauldI suppose a comment about why/when you would need to set ::pc/sym. I'm wondering if there is a way to define a resolver in a different namespace than the parser. Putting other-namespace/my-mutation in the app-registry didn't seem to work. Maybe I should try :refer 'ing it into that parser's namespace instead.#2019-01-0416:06pauldI'm going to test some things...#2019-01-0416:12wilkerlucio@pauld the result from defresolver / defmutation is a map, so in the end you have a var containing a map, what really matters at invocation time for mutations is the ::pc/sym, on mutations you always want to set unless you want to use the mutation fully qualified name (full definition namespace + name, that's the default setting if you don't provide one), makes sense?#2019-01-0416:17pauldyes, I think so - and the only time you don't want to use the default fully qualified name from the defmutation namespace is if you the fulcro-http-remote is looking up that defmutation via a different namespace than where the ultimate defmutation is defined and put into the registry.#2019-01-0416:27pauldHmm... still can't get a defmutation that is defined in another namespace to work. It's not critical, but this could cause people trouble because the fulcro book suggests separating mutations from reads and using Pathom will make them run into the same issue I had and my workaround is not the most obvious approach.#2019-01-0416:28pauldMy console gives me a transit decode error when I try this.#2019-01-0416:31pauldI'm either doing this wrong or we need to advise people to make sure they keep their mutations (and resolvers?) definined in the same namespace as the parser. And the workaround is to def the mutation in another namespace (ie mutations.clj) if you want to use it with the fulcro-http-remote that uses mutations.clj and mutations.cljs namespaces for the api calls.#2019-01-0416:46wilkerlucio@pauld can you send some sample code of the key points you are using? like the definition of the mutation, registry and call for it#2019-01-0416:46pauldI have my code on github, I'll push a branch and give you the link in a bit...#2019-01-0416:47pauldthe master will be my workaround#2019-01-0417:02pauldhttps://github.com/paulrd/cawala#2019-01-0419:18wilkerluciothanks for the repo, I'll take a look later today#2019-01-0417:03pauldmaster works. pathom-broken branch is more like I wanted to do but I get a json / transit error.#2019-01-0417:04pauld#2019-01-0503:19wilkerluciopathom 2.2.6 is released on clojars, this fixes a nasty bug on p/error-str that was returning a class type instead of string when the error didn't have message or data (which happens in NPE), this was breaking the transit output for some users, this version fixes it, thanks @pauld and @noxdeleo for the reports#2019-01-0503:31wilkerlucioAdd new section to the book about how to debug exceptions: https://wilkerlucio.github.io/pathom/#_debugging_exceptions#2019-01-0818:46tony.kayHm…bummer, @wilkerlucio when running in Debug mode the compiler generates a too large method for async-reader2, making it impossible to start a REPL.#2019-01-0818:46tony.kaycore async dumps a lot of code, and it is hitting the JVM limit on method size#2019-01-0818:46wilkerlucioyeah, and its not helping that I have a lot of macros that makes those even bigger#2019-01-0818:47tony.kayneeds to be split up, which would make it clearer probably as well šŸ™‚#2019-01-0818:47wilkerluciobut the solution is not so simple, because of how go blocks work, if I break the function I have to add more message passing, which is more overhead#2019-01-0818:47tony.kayyep#2019-01-0818:47tony.kaythem’s the cost of core async šŸ™‚#2019-01-0818:49wilkerlucioyeah, I'm already unsatisfied with the amount of overhead from it as current, I'm trying to reduce it so I would like to avoid adding more now, do you know if its possible to increase the max method code size?#2019-01-0818:50tony.kaynope, that is a hard limit in JVM#2019-01-0818:50tony.kay65536 bytes, if I remember right#2019-01-0818:50tony.kay16-bit size field in the JVM internals I think#2019-01-0818:58tony.kaydefinitely causing me dev issues…being able to set jvm breakpoints is important for certain bugs, and I can’t do it anymore because of pathom#2019-01-0819:07wilkerluciook, I can take a look on it, can you please open an issue?#2019-01-0819:13tony.kaysure#2019-01-0819:30wilkerlucio@tony.kay 2.2.7 should work šŸ˜‰#2019-01-0819:30wilkerlucio@souenzzo your updates to the diplomat are also in this release, thanks!#2019-01-1212:55souenzzoI'm doing a "pathom server" that calls a rest service via http driver I already have some statistics like - active sessions - user in active session (handled by cookies) Now I want to track and show - each query requested by each session - * each request done by each query Is com.wsscode.pathom.trace suitable to this job (focus in * )?#2019-01-1419:07souenzzoBUMP?#2019-01-1212:56souenzzoATM it's (not)persisted in-memory but in the future it may go to some storage.#2019-01-1419:07souenzzoBUMP?#2019-01-1419:42wilkerlucio@souenzzo about the trace thing, the trace is made to be generic, but at same time it accumulates at lot of data#2019-01-1419:43wilkerlucioif you wanna use it for analysis I suggest you run the processing/send part of it in some separated thread, so you don't slow down your users#2019-01-1420:13wilkerlucio@souenzzo one thing to remember is that the trace is just an event stream, a vector with a bunch of maps, this is lightweight to accumulate, but if you are using the tracing plugin, it has some post-processing steps that are expensive, just something to be careful with#2019-01-1421:42souenzzoOk. I will try to use and give some feedback about performance #2019-01-1716:49msshey all, are there any docs around about integrating pathom with the fulcro easy server?#2019-01-1918:27tony.kayDon’t use easy server…really. It is very simple to roll your own. The new template sets it all up for you.#2019-01-1717:36souenzzo@mss i think that it's underdocumented CLJS - When you create new-fulcro-client, you can pass (fc/new-fulcro-client :networking (net/fulcro-http-remote {:url "//localhost:8080/api"})) CLJ - Create a pedestal webserver in 8080 - create a route in POST/api. It should read and write transit - you can take all (parsed) body from /api and pass to your parser {:body (edn->transit (my-parser/parser {:my-db {...}} body)) :status 200}#2019-01-1717:37souenzzopedestal or any ring'ie. I know nothing about "fulcro server". Maybe can be easier use fulcro server and just exchange the parser part.#2019-01-1717:37mssfigured it out after a little tinkering — didn’t realize I needed to roll my own handler vs using the fulcro one out of the box. appreciate the help!#2019-01-1717:43souenzzo@mss from lein new fulcro my-project inside src/main/my_project/server_components/middleware.clj just remove (def server-parser (server/fulcro-parser)) And use (def server-parser (p/parser {.. pathom stuff ..}))#2019-01-1717:44souenzzoit should be way easier then create a new webserver... And should keep debug/support easier šŸ™‚#2019-01-1718:11Daniel HinesI’ve asked this same question myself. Thanks for this clear answer @souenzzo#2019-01-1719:06souenzzoPRO TIP: df/load has a great doc#2019-01-2003:15hmaurer@wilkerlucio I just watched your Conj 2018 talk, https://www.youtube.com/watch?v=yyVKf2U8YVg ; very interesting šŸ™‚ I have one question on your last demo involving the spacex and youtube APIs. You use ā€œaliasesā€ to make a connection between the launch video URL and a youtube URL, so that you can traverse the graph from the spacex data to the youtube data. What if, however, you had multiple youtube links? i.e. a launch video and a behind-the-scenes video. Is there a way to get to the youtube data for each of these?#2019-01-2006:54wilkerluciohello, thanks for taking the time to watch it. For the to-many would be very similar, you still do the alias in the same way, when we think about list processing, its no different from one item processing, its just the same thing many times, so if spacex had some list of videos, we could make so in pathom its a sequence of maps, example:
{:spacex.launch/all-videos 
 [{:spacex.launch/video-url "..."}
  {:spacex.launch/video-url "..."}
  {:spacex.launch/video-url "..."}]}
so in the previous example it has a list of items, which each has a video-url, so you alias this and that's it, makes sense?
#2019-01-2011:34hmaurer@wilkerlucio and thanks to you for taking the time to reply. Ha! I didn’t mean a list of videos. What I meant was two properties each pointing to a youtube url. For example, :spacex.launch/video-url and :spacex.launch/interview-url#2019-01-2011:34hmaurerFrom what you said about aliases in the talk, it seems that at this point Pathom would not be able to pick which video to follow#2019-01-2011:35hmaurerUnless there is some sort of join/other way around this issue#2019-01-2012:57wilkerlucioah, gotcha, well, you have some options in this case, in the example you sent now there not a single definitive way to link, so the api designer should choose, you could still point the video-url to translate to youtube as the default, and make a secondary join for the interview one, something like: {:spacex.launch/interview-youtube [:youtube.video/url]}, this way in the same level of the launch you get the main video, or join to this new name to get data about the interview part. You could make both depend on a join, this reduces the library reach but also reduces ability, the choice will depend on each case. A third option, in case there are two different variables that point to the same thing (maybe because some field got renamed for example), in this case you can implement resolvers for both cases, and pathom will choose which to use (and backtrack in case it fails, trying the other option)#2019-01-2013:15hmaurer@wilkerlucio ah, thank you! That’s useful information. So I take it there is no feature in EQL to ā€œalias an attribute to anotherā€ within a new context? I am not sure I am using the right terminology, but what I meant is that if, as per the example I gave, you have two youtube URLs in a context (assigned to different attributes), it seems like you should be able to write a query along the lines of ā€œI was to traverse down that road by assigning this attribute to :youtube.video/url within a new context, and get :youtube.video/name, and similarly for the other youtube URLā€#2019-01-2013:15hmaurerWithout a change in the API itself#2019-01-2013:16hmaurerWould such a query feature make sense? Or is there something I am missing? (i.e. would it not work because of X and Y, or…)#2019-01-2013:16hmaurerYour suggestion of changing the API design makes sense though, and if I have control over the API it seems sensible. I am just wondering if the approach I am suggesting above could be done#2019-01-2013:21wilkerluciothe API doesn't really have to change, you can just write more resolvers on top of it and make more nesting, the connection between apis is supposed to be mostly in user control#2019-01-2013:22wilkerluciopathom actually does have an alias feature to change the output name of things, but that doesn't affect the connect process, its just about renaming some output (the use case is when you need the same attribute twice using different parameters, so you can alias one of then to something else, you can do it as: (:my-key {:pathom/as :output-name})#2019-01-2013:27hmaurerAh, I see. Still have to get used to some of that way of thinking. Thanks!#2019-01-2014:36hmaurerAnother question on Pathom: would this be the right approach for a parametarised query?
{:pc/input #{:pagination/limit}
 :pc/output [{:user/all ....}]}
And how would one write a resolver where some of those inputs are optional? (i.e. a query that can have a bunch of filters, but where most filters are optional and have some defaults)
#2019-01-2014:49wilkerluciono, inputs and params are two different dimensions, the input is more like information to traverse, a dependency (a requirement of the operation to run, that might be available via some computation), its a blur line but from experience I can tell you pagination is usually a param, not an input, so the param you can declare using ::pc/param (be careful, your code is using a single : on the namespace, you need o use :: to expand the alias to get the real keyword, if you not familiar with fully qualified keywords check: https://www.deepbluelambda.org/programming/clojure/know-your-keywords)#2019-01-2014:50wilkerluciothen, you can read the params with: (-> env :ast :params)#2019-01-2014:50wilkerluciothe ::pc/param use the same syntax as ::pc/output, currently it does nothing to be honest, but in the future it will be used for auto-complete on the codemirror query editor#2019-01-2014:50hmaurerah yes, that was just a typo for ::. and thanks for the information on params. Can they be optional as well?#2019-01-2014:58hmaurerah nevermind, the doc seems to indicate there is no validation at the moment#2019-01-2014:58hmaurer(as you just said as well)#2019-01-2015:00hmaurer@wilkerlucio what is the syntax for passing params to a resolver? the doc only seems to mention params in conjunction with mutations#2019-01-2015:01wilkerlucio@hmaurer you can find some docs for it in the EQL spec: https://github.com/edn-query-language/eql#parameters#2019-01-2015:02wilkerlucioand you motivated me to write the long missing docs about params in the pathom book, doing it now šŸ˜‰#2019-01-2015:03hmaurer@wilkerlucio šŸ˜„#2019-01-2015:04hmaurer@wilkerlucio unrelated but coming from GraphQL (which I use on some hobby projects) I particularly liked the part of your talk on interfacing multiple domains (i.e. spacex + youtube). That was a big questionmark for me in GraphQL; how to do schema stitching. Pathom’s attribute-level approach feels a lot more natural for this#2019-01-2015:19hmaurer@wilkerlucio is there a section of the documentation that discusses how to best represent ā€œsoftā€ errors? (such as a validation error on a mutation). I was just reading https://wilkerlucio.github.io/pathom/#_error_handling which seems more suitable for hard, non-user-caused failures#2019-01-2015:19hmaurer(I am using Fulcro)#2019-01-2015:21wilkerluciothere is nothing fully done for it, but you can expose extra data in your errors, pathom supports a ::p/process-error fn that you can use to customise the output, what I usually do is expose a map so I can attach extra info, so if you have validations that can only be done on the server, you can add something to error to tell the UI to represent accordingly#2019-01-2015:34hmaurer@wilkerlucio ah thanks! Do you know if this works in conjunction with Fulcro’s fallbacks? (http://book.fulcrologic.com/#Fallbacks). Either way I’ll try it right away#2019-01-2015:35wilkerlucioit does, the fallback will cover you from network errors (500, network timeout, etc...), so not a lot of info available, but at least you can tell the user#2019-01-2015:36hmaurer@wilkerlucio ah, but I meant as a way to pick up the validation error and transact some state (i.e. an error flag + message on the approriate field)#2019-01-2015:37hmaurerwell anyway I’ll try this out šŸ™‚#2019-01-2015:37wilkerluciothe fulcro fallback will only trigger if the remote responds with errors, pathom will never respond with an error (unless something had fail big time)#2019-01-2015:38wilkerlucioso, I'm just getting you aware that to handle the "soft" errors from pathom, the best thing you can do now is using the pessimistic mutations from incubator: https://github.com/fulcrologic/fulcro-incubator#pessimistic-mutations#2019-01-2015:38wilkerluciothis way you will have a nice way to handle those#2019-01-2015:39wilkerluciothe fallback is still useful, but for other reasons (major failures, like having no internet to hit the parser endpoint, or something crashing bad)#2019-01-2015:39hmaurerah ok, makes sense.#2019-01-2015:55wilkerlucioAdd new docs for how to use parameters in resolvers https://wilkerlucio.github.io/pathom/#_parameters @hmaurer#2019-01-2016:56hmaurerthat was fast! thanks šŸ™‚#2019-01-2316:56Daniel HinesAre there any examples of consuming pathom from JS?#2019-01-2317:52souenzzo@d4hines you say JS as browser or JS as node? it's CLJS or "raw JS"? If you have a pathom API in /api you can do fetch("/api" {method: "POST", body: "[:api/hello]"}).then(x => x.text()).then(console.log) it will print "{:api/hello \"Hello\"}"#2019-01-2317:54Daniel HinesOh, ok, I assumed transit was built in or something. That still leaves how to deal with namespaced keywords and other impedance mismatches between EDN and JSON, but that’s not unique to pathom.#2019-01-2317:57souenzzoyou can also run pathon on client with something like parser(ctx, edn.read_string("[:api/hello]")).then(edn => console.log(edn.to_js(edn))) will print {api_hello: "Hello"} There is really many ways to use pathom.#2019-01-2317:59Daniel HinesOk, that last example pretty well solves it for me. Thanks!#2019-01-2318:01souenzzoI never see no one using pathom like this. But I'm sure that it's possible.#2019-01-2318:03Daniel HinesI want to show the capabilities of pathom to co-workers, but they aren’t ready for Clojure yet šŸ˜€#2019-01-2319:52msshey all, trying to get a simple mutation up and running and my symbols generated by the defmutation macro don’t seem to be getting resolved by the connect plugin registrar. code looks like the following:
(pc/defmutation test-mutation [env params]
  {::pc/sym    'test-mutation
   ::pc/params [:test-tempid]
   ::pc/output [:test-id]}
  (let [test-id (.toString (java.util.UUID/randomUUID))]
    (println "IN MUTATION")
    {:test-id test-id}))

(defn make-parser []
  (p/parser
    {::p/env {::p/reader               [p/map-reader
                                        pc/parallel-reader
                                        pc/open-ident-reader
                                        p/env-placeholder-reader]
              ::p/placeholder-prefixes #{">"}}
             ::p/mutate pc/mutate
             ::p/plugins [(pc/connect-plugin {::pc/register [test-mutation]})
                          p/error-handler-plugin
                          p/request-cache-plugin
                          p/trace-plugin]}))
and calling
((make-parser) {} `[(test-mutation {:test-tempid "12345678"})])
=> Mutation not found
#2019-01-2319:53mssany idea why that might be?#2019-01-2319:59mssfigured it out. didn’t realized that adding the ::pc/sym was adding an ns onto the mutation sym#2019-01-2320:02Daniel HinesThe current namespace, right?#2019-01-2320:02Daniel Hines(i.e namespace in which test-mutation was defined)#2019-01-2320:05mssyep exactly#2019-01-2320:06mssunfortunately when defining the mutation in the data format like so:
(def my-mutations
  [{::pc/sym 'test/test-mutation
    ::pc/params [:test-tempid]
    ::pc/output [:test-id]
    ::pc/mutation (fn [env params]
                    (let [test-id (.toString (java.util.UUID/randomUUID))]
                      (println "IN MUTATION")
                      {:test-id test-id}))}])
and running a parse like:
(my-p {}
      `[(test/test-mutation {:test-tempid "12345678"})])
I’m getting an error " Mutation not found - {:mutation test/test-mutation}"
#2019-01-2320:06mssseems like there’s some disconnect between the way the macro does symbol resolution for mutations and the data literal format#2019-01-2320:08mssweird because the output of the mutation data literal looks like:
#:com.wsscode.pathom.connect{:sym test/test-mutation, :params [:test-tempid], :output [:test-id] ...}
#2019-01-2320:15mssunfortunately when defining the mutation in the data format like so:
(def my-mutations
  [{::pc/sym 'test/test-mutation
    ::pc/params [:test-tempid]
    ::pc/output [:test-id]
    ::pc/mutation (fn [env params]
                    (let [test-id (.toString (java.util.UUID/randomUUID))]
                      (println "IN MUTATION")
                      {:test-id test-id}))}])
and running a parse like:
(my-p {}
      `[(test/test-mutation {:test-tempid "12345678"})])
I’m getting an error "Mutation not found - {:mutation test/test-mutation}"
#2019-01-2412:11wilkerluciostill having the issue mms? I wonder if the test might be expanding to somethign else#2019-01-2414:26hmaurerHi! I am trying to use placeholders but for some reason I can’t get them to work. Here is the first query I tried:
[{:current-user [:user/id :user/first-name]}]
which returned
{:current-user
 {:user/first-name "Henri",
  :user/id #uuid "2d3f8164-9a7c-45b6-b0ef-194cfd5b84be"}}
Then I tried:
[{:current-user [:user/id {:>details [:user/first-name]}]}]
which returned:
{:current-user
 {:user/id #uuid "2d3f8164-9a7c-45b6-b0ef-194cfd5b84be", :>details {}}}
#2019-01-2414:26hmaurerwhat did I do wrong?#2019-01-2414:32hmaurerAh, got it. The syntax is not :>details but :>/details!#2019-01-2415:42wilkerlucioyup, > is considered the special thing, and this is configured as part of your environment#2019-01-2415:43wilkerlucioyou can even use a different one, but unless the default > is causing you trouble (colliding with something else) I encourage you to keep using the default#2019-01-2515:13Andreas LiljeqvistHow should I structure resolvers for something that can return a lot of different combinations of attributes? [{[:person/id 1] [:db/id :name :fifty-more-keys-that-might-be-present-in-a-query]}] Listing all possible in [::pc/output []] could work, but then a lot will be returned with :pathom.something/not-found#2019-01-2515:20Andreas LiljeqvistPerhaps it is the right thing to do and elide in the post-process parser#2019-01-2515:21wilkerlucio@andreas862 if what you have is some heterogeneous list (having multiple "types" of return, were the attributes are consistent per type) you can use Union Queries https://github.com/edn-query-language/eql#unions / https://wilkerlucio.github.io/pathom/#_union_queries#2019-01-2515:24wilkerlucioconnect supports you specifying output declarations using the union feature, one caveat is that the tools doesn't support yet (for auto-complete) but the processing works#2019-01-2515:24wilkerlucioplease let me know if that looks right for your case and if you need any help implementing it, might be a good time to get more docs in šŸ™‚#2019-01-2515:36Andreas LiljeqvistChecking your link, but don't know if it is exactly what I want. Trying to specify resolver for an entity where the query is some subset of possible attribs. Exhaustive defresolver for all combinations probably isn't right. Just an ::pc/output all-possible-attribs could be right if I elide not-found. Summary - I probably know too little Pathom to think about the problem in the right way#2019-01-2515:37wilkerluciook, let's try to make sense of it šŸ™‚#2019-01-2515:38wilkerlucioone thing to consider when splitting/merging resolvers is that how much work is been done#2019-01-2515:38wilkerluciofor example, if you are reading data from some service that returns 5 attributes (or 10, or 100...), since you already have the data, it doesn't make sense to separate it#2019-01-2515:38wilkerluciobut computed data must be in a separated resolver most of the times#2019-01-2515:38wilkerluciothis way you can optimize how much computation is been done#2019-01-2515:39wilkerluciothe output will have only what the user asks, so no unnecessary bandwidth will be used#2019-01-2515:41wilkerluciofrom what kind of you source you are pulling the data?#2019-01-2515:42Andreas LiljeqvistDatomic - so basically a thin wrapper over (d/pull (db) query eid)#2019-01-2515:42wilkerlucioyeah, I personally never did a parser using datomic directly some people here did, in general what I see then doing is making a pull like you#2019-01-2515:42wilkerlucioand relations can be defined using new resolvers#2019-01-2515:43wilkerlucioits ok to output a lot of things from the resolver, and datomic been all in memory (cached) also helps#2019-01-2515:47wilkerluciowhen some resolver exposes a map, that map is merged in what we call "current entity", this makes that attributes to be locally cached for future access, but they are only exposed if the user asks for then, this means you don't have to manually elide the results, only the requested data will be exposed in the final output#2019-01-2516:10Andreas LiljeqvistThis is when requesting attributes that aren't guaranteed to be present. Can I elide not-found on specific resolvers? Thanks for your help#2019-01-2516:11wilkerlucioah, yes, for eliding not found is quite easy#2019-01-2516:11wilkerlucioyou can use a post-process plugin: ::p/plugins [... (p/post-process-plugin p/elide-not-found)] something like this#2019-01-2516:15Andreas LiljeqvistAlready using it šŸ™‚ I wanted to know if I could bless specific resolvers instead of global, but I don't think I really have a reason to specify#2019-01-2516:15Andreas LiljeqvistSo ignore - and thanks#2019-01-2517:08hmaurer@wilkerlucio Hello šŸ™‚ So as I mentioned the other day I watched your talk on Pathom and have been playing with it yesterday. I have used GraphQL in the past and noticed that Pathom seems a lot neater for extending the knowledge graph than GraphQL is (i.e. adding the ability to fetch youtube data, as per your demo, etc). However I am curious. With your experience writing/using Pathom, what do you think the tradeoffs are? (if you are familiar with GraphQL). As in, do you think Pathom is a straight up better/more flexible way to fetch data and commit mutations? Do you think it is lacking some features that GraphQL has? Do you think they don’t address exactly the same issues?
#2019-01-2517:17wilkerlucio@hmaurer hello šŸ™‚ So, lets talk about the constrast, if you did read already I like to mention the differentes I already wrote about in EQL docs: https://github.com/edn-query-language/eql#graphql-comparison#2019-01-2517:18hmaurerAh! I was trying to find a written comparison in Pathom’s doc. I should have checked EQL’s doc as well#2019-01-2517:18wilkerlucioif I try to describe the big picture in short, the major difference is about the decision of what are the building blocks#2019-01-2517:19wilkerlucioGraphQL is all based on types/containers, the primary building blocks are the types, they are referenced all the time in the whole system#2019-01-2517:19wilkerluciowhile pathom uses a more datomic/spec like mindset, where the attributes are the building blocks, not the types/containers#2019-01-2517:21wilkerlucioand this mindset is what makes the things from pathom possible#2019-01-2517:21wilkerlucioby having the attributes as first class we can talk about then without ever have to reference (or name) aggregates#2019-01-2517:22wilkerlucioIMO at this point the major trade-off is the small community (compared to GraphQL), that in consequence means less tools, less people finding bugs... so it goes in a slower pace of change, but that can be good as well šŸ˜›#2019-01-2517:25hmaurer@wilkerlucio Thanks for the detailed reply šŸ™‚ The community is always a trade-off for newer projects though! Do you think there is a technical tradeoff in the choice of being attribute-centric?#2019-01-2517:26wilkerluciomore that I try it, more I like it, if you watch Rich's presentations, he talks about that all the time, that's clojure foundations, maybe we miss on tooling, but that's more about things not been ready#2019-01-2517:31hmaurer@wilkerlucio it’s interesting; this is the kind of experiment that would not be as natural in, say, javascript. It feels like namespaces on keywords are a key feature (although afaik Pathom doesn’t care about namespaces as such)#2019-01-2517:35wilkerlucio@hmaurer on the contrary, namespaces are very important for Pathom, and for attribute modeling in general I think#2019-01-2517:35wilkerluciowithout that you get only gigantic tedios to type things all the time (remember C)#2019-01-2517:36hmaureroh, how come? Could you write :user-email instead of :user/email and get essentially the same results?#2019-01-2517:36hmaurer(although of course namespaces are a lot nicer)#2019-01-2517:36wilkerlucioyeah, we also use it for some features, the placeholder nodes for example, they are based on namespaces#2019-01-2517:37wilkerlucioone thing that I see as underused in the clojure community is the namespaced keywords, its getting better after spec, but I like to see more, I think keywords should have NS as much as your code has to have#2019-01-2517:37wilkerluciothe same reasons we use to validate namespaces for code (so we can use multiple functions that have the same name without issues), the same is valid for keywords IMO#2019-01-2517:37hmaurerMaybe I misexpressed myself. What I meant to say is that it seemed to me namespaces aren’t essential for Pathom (in the sense that you could write :user-email instead of :user/email), but they are essential for this kind of attribute-driven approach as they allow you to structure attributes (group user attribuets, etc) in a way that is more organised than key prefixes#2019-01-2517:38wilkerlucioyeah, agreed, but they really help on the mindset#2019-01-2517:38hmaurerYeah, definitely. That’s what I was trying to say; a project liek Pathom likely wouldn’t arise in JavaScript simply because it doesn’t support key namespaces#2019-01-2517:38wilkerlucioI find hard to convince a JS programmer that: {"youtubeVideoId": "..."} is better than {"id": "..."}#2019-01-2517:38hmaurerIt wouldn’t be as ā€œnaturalā€#2019-01-2517:40hmaurerHow do you tend to handle security in a Pathom API? (I had the same question about GraphQL). Do you do it at the resolver level?#2019-01-2517:40hmaurersecurity => access control#2019-01-2517:41wilkerlucioyeah, resolver level sounds right, so you can make it as specific as you need (up to each attribute)#2019-01-2517:42wilkerluciothere is a cool (pending docs) feature in pathom that allows you to write interceptors (kind like pedestal) for resolvers#2019-01-2517:43wilkerluciothis way you could make some generic security plugin and wrap resolvers with then as you see fit#2019-01-2517:43wilkerluciolooks like this:#2019-01-2517:44hmaurerOoh, that sounds very useful. So that feature is already out, just not documented?#2019-01-2517:44hmaurerAlso, unrelated but do you know https://www.graphile.org/postgraphile/ ?#2019-01-2517:45wilkerluciothats nice, and since we can "pull" graphqls into Pathom, taht could be used strait out šŸ™‚#2019-01-2517:46wilkerluciocaveat: there are some limitations on graph to graph requirements at this point, but they are been worked out#2019-01-2517:47hmaurer@wilkerlucio yep, I read that you can ā€œpullā€ graphql APIs right into Pathom, however I linked it because I tried to contribute to this project but the codebase is a bit of an entangled mess of ā€œpluginsā€ to extend the GraphQL API. From the little I know about Pathom I think it should be a lot more natural to implement#2019-01-2517:47hmaurerI might give that a go after my spring exams; it would be extremely useful#2019-01-2518:40hmaurer@wilkerlucio out of curiosity, what are the graph to graph m=limitations?#2019-01-2519:07wilkerlucio@hmaurer mostly that inputs currently have to be flat, this is a problem to request graphql for example, in the future I want the library to support inputs like: ::pc/input #{{:github.viewer #{:github.user/name}}}#2019-01-2519:10hmaurer@wilkerlucio ah, I see. by the way, unrelated question: do you currently support documentation on Pathom attributes? And introspection on said docuemntation?#2019-01-2519:10hmaurer(like graphql does)#2019-01-2519:26wilkerluciono docs directly from pathom, but specs are getting docstrings, so you can just use that#2019-01-2519:28hmaurer@wilkerlucio oh are they? I hadn’t heard, that’s great. Does Pathom exposes specs someone through introspection queries?#2019-01-2519:28wilkerlucioabout tooling, I've started writing an index explorer last week, I wanna try some ideas on how to visualize and explore the index, so you can search for attributes, and see how to get it, what it offers, specs, examples (from spec generators), maybe some d3 interactive thing, I dunno yet šŸ™‚#2019-01-2519:28wilkerluciono, its your job to have the specs on the client, since they are code I can't garantee transmission#2019-01-2519:29hmaurermakes sense#2019-01-2517:44wilkerlucioyup, I'm getting an example for you#2019-01-2517:48wilkerluciothis is what transform usage looks like:#2019-01-2517:48wilkerlucio
(pc/defresolver my-resolver [_ _]
  {...
   ::pc/transform (fn [resolver-map]
                    (update resolver-map ::pc/resolve
                      (fn [resolve]
                        (fn [env input]
                          (let [res
                                (resolve
                                  (assoc env :modified "env")
                                  (assoc input :input-extra "change-input"))]
                            (assoc res :change-output "too"))))))})
#2019-01-2517:48wilkerlucioso you can generative what goes in the ::pc/transform#2019-01-2517:49wilkerluciohere you can find some built-in transform functions: https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/connect.cljc#L1244-L1275#2019-01-2517:49hmaurerOh, interesting#2019-01-2517:49hmaurerthanks for that#2019-01-2800:11hmaurer@wilkerlucio totally unrelated to the discussion happening on #fulcro but since we had a chat the other day about the differences with GraphQL: you can provide some data about a node that ā€œdoesn’t existā€ in any persistence layer and derive more data#2019-01-2800:11hmaurerthis seems super handy#2019-01-2800:12hmaureri.e. when creating a new entity you can query and display derived data, even though the entity hasn’t yet been persisted in the database. This, afaik, isn’t possible with GraphQL#2019-01-2801:16wilkerlucio@hmaurer correct, this is mostly a result of the mindset, there is no real "entity", its just labeled data aggregated in maps, its just available or not, if is you can move on#2019-01-2801:18wilkerluciobut you noted an interesting property that are not obvious and its good to call it out, thanks for the reminder šŸ™‚#2019-01-2814:58hmaurerHello @wilkerlucio ! If you remember, a couple of days ago I asked you a question after watching your Conj 2018 talk. I had found your SpaceX <-> Youtube example very cool but wondered how one would deal with multiple properties all pointing to youtube videos (i.e. :spacex/launch-video, :spacex/interview-video, etc). The solution for a single property was to use an alias, but there didn’t seem to be an obvious, clean solution to using multiple properties, other than extending the API or structuring it differently. I have had a bit more time to think about it since then, which is why I am writing this now šŸ™‚ As far as I understand, aliases work by, well, aliasing a field in the current ā€œcontextā€ (is that the right term). So with your youtube example above, you could alias :spacex/launch-video to :youtube/url, and then youtube resolvers could go from :youtube/url to more information, such as the video title, etc. Now, here’s my issue with this. Let’s say you are in the context of a ā€œspace launchā€ (to stay on the spacex example). This space launch has a :spacex/interview-video property, which contains a youtube URL. You want to fetch more data about that video, so you alias it to :youtube/url and then query for :youtube/name. Now you have a :youtube/name property on the space launch context. But that property isn’t a property of the space launch, it’s a property of the interview. It seems like the property is now on the wrong node. And this creates the further issue that you can’t both get the youtube video name for the interview AND the launch video. Wouldn’t it make more sense to have a sort of join/alias hybrid? Let’s say I would write {{:spacex/interview-video :youtube/url} [:youtube/name]} (made up syntax). This would create a new ā€œcontextā€ with :youtube/url within that new context set to the value of :spacex/interview-video. You could then continue to explore the graph from there. Right now I only thought about the one-property use-case, but perhaps it would make sense to be able to pull as many properties as you wish from the parent into this new context.#2019-01-2814:59hmaurerWhat do you think? Would this make sense? Or am I ranting while missing an obvious issue?#2019-01-2815:00hmaurerTL;DR; with my limited understanding of Pathom using an alias for the spacex example you gave feels like a hack; you end up with properties on wrong entity/node.#2019-01-2817:24wilkerlucio@hmaurer yes, if you have multiple connections then having a join for each makes more sense, that was mostly a case that I wanted to show people whats possible when there is no ambiguity, but in the spacex case, since a video might have more than one relationship to a video, its better to make each a separated join to remove ambiguity#2019-01-2819:01hmaurer@wilkerlucio what I am wondering is whether you could have this ability at query time, instead of having to extend the API, if that makes sense#2019-01-2819:04wilkerlucioI don't think its a good idea to do it at query time, starts to add a lot of complexity while we already have something that works and its consistent, I believe you can do something like this yourself writing plugins, but I don't see that been a core feature at this stage#2019-01-2819:11wilkerlucioI like to avoid adding features as much as possible at this stage, keeping it simple will make it evolve better, so unless we found some situation that really can't be accommodated then would be a time to think more about it šŸ˜‰#2019-01-3022:43hmaurer@wilkerlucio Hello šŸ™‚ Quick question: do you often/ever do ā€œcontext sensitiveā€ edges? e.g. say you have a system with ā€œexercisesā€ (global for all users) and ā€œexercise solutionsā€ (one per exercise per user). Would you add a :exercise/solution property which points to the current user’s solution for a given exercise?#2019-01-3109:22souenzzoI'm also not sure about that but I always try to avoid.#2019-01-3112:11wilkerlucioI would use :exercise/user-solution to this specific link, so the name has the correct meaning#2019-01-3122:16ecasparyOpening I find Pathom to be quite fascinating; thanks for creating it and making it freely available! I’ve been writing resolvers for some internal services at work and am quite pleased with the results. Questions I’m have some questions regarding recursive pull queries: are they supported? eg.
;                                                 v
 (d/pull db '[:node/id :node/mass {:node/neighbors 2}] ent-id)
It seems like Pathom provides all the tools to handle a query like the one above. Is that accurate? If I wanted to support bounded recursion, what must I do as the author of a resolver? Or should I be writing a custom reader? Distilled Example Context App domain is inherently graph-like. We prototyped in Datascript. Pull + bounded recursion provided a natural way to explore limited regions of this graph. To learn Pathom, I’m trying to recreate this graph API experience on the production system (where we aren’t using datascript / datomic). Closing Thanks for reading; thanks for Pathom.
#2019-01-3122:19hmaurerI can’t answer your question but props on structuring it so well šŸ˜„#2019-02-0101:24wilkerlucio[com.wsscode/pathom "2.2.9"] is out! For everyone using Pathom I highly suggest to upgrade, this version has some fixes we encountered recently to calculate paths on some complex situations (where it goes from one to many and back, hard to explain but better have it working properly). This also adds p/pre-process-plugin.#2019-02-0101:27wilkerlucio@ecaspary welcome šŸ™‚ about recursive queries, yes they are supported, there is not much you have to do, they should work out of the box even you a valid recursive link in your resolvers, your example looks almost correct, but the :node/neighbors will not work that way#2019-02-0101:28wilkerlucioit can be fixed using:#2019-02-0101:28wilkerlucio
(def mock-db
  {:node/id {0 {:node/id 0 :node/mass 0.00 :node/neighbors [{:node/id 1}]}
             1 {:node/id 1 :node/mass 1.11 :node/neighbors [{:node/id 2}]}
             2 {:node/id 2 :node/mass 2.22 :node/neighbors [{:node/id 3}]}
             3 {:node/id 3 :node/mass 3.33 :node/neighbors [{:node/id 0}]}}})
#2019-02-0101:28wilkerlucioso the elements are now maps that represent a context, pathom can pick up on that#2019-02-0101:29wilkerlucioif I may, you could start simpler to experiment using to-one connections, might be easier to understand at first, to be honest I dont remember trying recursion to many like this#2019-02-0101:32wilkerlucioI just tried running it, it works for to-one, but not for to-many, I'm interesting to hear about your case on recursive queries to-many, are those valid in datascript?#2019-02-0101:34wilkerlucioso, this works with this mock data:#2019-02-0101:34wilkerlucio
(def mock-db
  {:node/id {0 {:node/id 0 :node/mass 0.00 :node/neighbors {:node/id 1}}
             1 {:node/id 1 :node/mass 1.11 :node/neighbors {:node/id 2}}
             2 {:node/id 2 :node/mass 2.22 :node/neighbors {:node/id 3}}
             3 {:node/id 3 :node/mass 3.33 :node/neighbors {:node/id 0}}}})
#2019-02-0101:34wilkerluciobut :node/parent would be a better link name in this case#2019-02-0113:33danielstocktonIs walkable a somewhat vetted and recommended way of dealing with sql + pathom?#2019-02-0115:49wilkerlucio@danielstockton what you mean by vetted?#2019-02-0115:50wilkerluciowalkable works fine with pathom, but Walkable doesn't support connect at this stage, so mixing those things don't work well#2019-02-0115:54danielstocktonJust wondering if those that use pathom usually use walkable, and if the two projects are fairly well in agreement (or if there are contentions or overlap)#2019-02-0115:54danielstocktonIf they're likely to diverge or significantly overlap, i'd rather pick one#2019-02-0115:55wilkerlucioI personally thing the connect infrastructure is what enables the things I create to grow, scale and connect across different sources, but it will depend on the use case, if you have just a lot of SQL maybe use Wakable, but you can use connect and just write resolvers that use some SQL library, and with that you get all the connect features (path resolution, tracer, auto-complete...)#2019-02-0115:56wilkerluciowould be great if Wakable did support connect, it could be done, it needs to generate proper connect indexes, and then you could mix both things#2019-02-0115:56danielstocktonUnderstood, answers all my questions, thanks#2019-02-0213:56hmaurer@wilkerlucio more of a Pathom question so I’ll post it here: in my application I’ve found myself using :current-user as a root for pretty much everything, since all my data is user-dependent. Is that sensible?#2019-02-0213:56hmaurerSo I have html5 routing set up, and on route change I load some query rooted at :current-user#2019-02-0213:56hmaurerpulling data for the page#2019-02-0213:59wilkerlucioyup, IMO you should avoid simple keywords entirely#2019-02-0213:59wilkerluciobecause at some point systems will connect, if they both use simple keywords they can't talk without ambiguity#2019-02-0213:59hmaurerwhat do you mean? I should use :my-app/current-user?#2019-02-0214:00wilkerluciothats a good topic for a documentation section, but yes, you app/company should prefix everything with its name#2019-02-0214:01wilkerlucioso would be :my-app/current-user :my-app.user/name ...#2019-02-0214:01hmaurerso even entities, instead of :user/first-name have :my-app.user/first-name.#2019-02-0214:01hmaureryep#2019-02-0214:02hmaurerthat makes sense for inter-system communication#2019-02-0214:02wilkerluciosimple keywords are only good enough on demos, which is a problem because people copy from demos, heh#2019-02-0214:03wilkerlucio@hmaurer also having qualified keywords allow you to get more data around it, because they are context free information, spec been the major example case for extracting more info about the keywords thenselves#2019-02-0214:06hmaurer@wilkerlucio yeah that makes perfect sense, although in my current project at some point I over-qualified keywords I think. I started qualifying them by module as well, e.g. :my-app.my-module.some-entity/some-property and they got very long. Perhaps the app prefix is enough#2019-02-0214:06hmaurer(I was doing this for specs)#2019-02-0214:10wilkerlucioyeah, remembering that what matters is the name, the namespace is just a qualifier for the name to avoid collision#2019-02-0214:10wilkerluciosome people can argue using domain names (like Java does) but in practice I see project name been enough#2019-02-0214:11wilkerluciofor example, if we make a bunch of resolvers for youtube, that will hardly crash with anything else (unless someone is doing on purpose, not accidental)#2019-02-0214:47pvillegas12Switching over here, as this is more pathom specific. For pathom, I see the most usefulness to manage multiple sources of data that do not implement a graphQL-like attribute level of querying. With Datomic, as we have pull syntax, we can pass most of Fulcro queries directly to a single resolver that hands of the query to d/pull#2019-02-0214:49pvillegas12My question is: Given Datomic has pull, what is a good way to combine resolvers given we already have arbitrary query expressions in the db?#2019-02-0215:01pvillegas12(I know in that expressing my API with pathom is the way to go, as I can later expose a graphQL API to talk to my data)#2019-02-0215:01wilkerlucio@pvillegas12 I'm afraid I can't help much with that because I lack experience using datomic with pathom, for what I see people tend to avoid exposing they whole datomic tree via pull, if you have that concern in mind then breaking it down and controlling whats acessible ends up been an asset, but if you are in a situation where uncontrolled access is ok (maybe something internal dev tool) then I think will be easier to use datomic entities instead of pull, maybe you will need a custom reader, but from that you could expose everything and combine with resolvers for computed values#2019-02-0215:01wilkerlucioabout the graphql out, that's something I want to do, but its not there yet, pathom can consume graphql apis, but not expose it#2019-02-0215:02pvillegas12why would they avoid exposing the whole tree?#2019-02-0215:03wilkerluciobecause the UI has direct access to it, so it could leak private data, since all connections would be exposed#2019-02-0215:04pvillegas12you mean when you introspect it?#2019-02-0215:04wilkerlucioor just protected data in general, delegating all to datomic pull gives you no control, so you dont want your end user to have direct access to it#2019-02-0215:04pvillegas12I was thinking more of datomic pull + policy logic in the resolver env#2019-02-0215:05pvillegas12Let’s see if I understand this, you’re saying create a resolver that does a d/pull and then do a select-keys without the private.data/ namespace for example?#2019-02-0215:06wilkerluciocorrect, that's the point, you have to be in the middle, you cant blindly send it over, but I'm just speculating possibilities that you could try, as I told I have no experience using datomic with pathom#2019-02-0215:06pvillegas12yeah, this conversation is useful either way for me šŸ˜„#2019-02-0215:06wilkerlucioyes, but think about nested queries, then can get complicated to "select-keys"#2019-02-0215:06pvillegas12correct#2019-02-0215:07wilkerlucioso what I see people doing is providing level 0 and level 1 nesting on their pull patterns#2019-02-0215:07wilkerluciolike: [:user/name {:user/addresses [:address/id]}]#2019-02-0215:08wilkerlucioso no more than 1 nesting level, and leave the "ids" in place so other resolvers will pick up an continue#2019-02-0215:08wilkerluciowith this pattern each resolver level have control over "one resource" kind, and you can split more depending on how much control you want to have#2019-02-0215:10pvillegas12Got it, so the resolver specifies which keys you want to expose publicly essentially#2019-02-0215:10wilkerlucioyup#2019-02-0507:23eoliphantHey guys. quick question. I’m using pathom to ā€˜smooth out’ the interface to data coming back from datomic. In general pretty straight forward, even have some generic query funcs, that my pathom resolvers call that mostly obviate the need for per resolver db queries, etc. I’m currently trying to figure out the best way to flatten datomic ā€˜enums’ generically. so say my pathom query is [... [:my-enum-attr]] and I expect to get {:my-enum-attr :enum-val/one}. Of course datomic returns something like {:my-enum-attr {:db/id xxx :db/ident :enum-val/one}}. I was thinking about the stuff in the manual, around managing derived attributes with resolvers, but I don’t think that works here since I already have the attribute I want, I just want to apply a transform to its value, without having to do something more brute force with the results prior to returning it to pathom#2019-02-0510:49wilkerlucio@eoliphant bom dia šŸ™‚ I think a good idea would be to create a new name for the "normalized" version of the enum, for example, if you have :animal/specie thats something like :animal.specie/dog, then you could have a resolver for the attr :animal/specie-ident that converts and extracts, having a separated name helps to avoid shadowing the original name and making it unaccessible. With this basic idea in mind, you could try to inspect the datomic schema and generate those automatically, I don't remember what the datomic schema looks like after commited, but I think worth checking, it might have enough information to automate those enum resolvers#2019-02-0513:38eoliphantbom dia šŸ™‚ just got back, dying from the cold lol. Ok, that’s interesting. I’d actually already added a bit of a hack,. I’ve been working on a tool that lets me define a sort of generic domain/attr model, so I can create docs, gen datomic schema, warn if specs are missing etc. So I already ā€˜know’ that :animal/specie is a ā€˜enum type’. SO used that to look at the query and see if I’d need to do any munging of the result. I’ll use the same info to try what you suggested.#2019-02-0513:38eoliphantobrigadao šŸ™‚#2019-02-0517:17Daniel HinesI've thought that it should be totally feasible to create a datascript/datomic -> pathom plugin like the existing graphql one, no?#2019-02-0517:19wilkerlucio@d4hines totally, just never got prioritised on my side, but I think would be a nice addition#2019-02-0517:25Daniel HinesI wouldn't mind taking a stab at this, starting with Datascript. I'd like to play with your graphql examples to get a feel for it. Are your examples from the Conj on GitHub?#2019-02-0517:32wilkerluciojust pushed at https://github.com/wilkerlucio/conj2018demo#2019-02-0517:32wilkerluciodidn't tested, please let me know if there is anything broken#2019-02-0518:01Daniel HinesThanks!!#2019-02-0710:12liesbethHi! I was looking at connecting graphql as a fulcro remote. It seems to load the schema now, but I cannot explore what resolvers/queries are now available, does anyone have suggestions on how to find that out?#2019-02-0710:41liesbethI found a solution! Probably the rest of the world already knew this, but if you press the grey circle left of ā€˜Run query’ in the Fulcro Inspector Query tab, it will give you suggestions on what you can query šŸ™‚#2019-02-0712:06wilkerlucio@liesbeth exactly, maybe we should somehow try to make that more obvios, in the first versions it used to load it automatically, but that was a problem for people not using pathom#2019-02-0712:06wilkerluciojust not sure how, but I'm open to design ideas šŸ™‚#2019-02-0712:07liesbethOkay. If I think of anything I’ll let you know šŸ˜‰#2019-02-0712:09liesbethI actually also have another question: in the documentation about graphQL it says ā€ Name munging is customizable via Pathom settings.ā€ How would that work? I am using a graphql endpoint that has query and mutation names capitalized, and am not sure how to change this is pathom.#2019-02-0712:17wilkerlucio@liesbeth trying to remember, I know there is a flag for that, try setting :com.wsscode.pathom.graphql/js-name (its a fn that receives a string, you can use identity for no change) in the graphql settings#2019-02-0712:19wilkerluciohumm, but looking at the sources I'm not sure if thats true for the connect integration, we may need to change some code to fix that, the js-name works on the simple integration, but it should work for connect too, just seems you are first to stumble on it#2019-02-0712:20wilkerluciocapitalized is the default in graphql, pathom converts it to kebab case to be more clojure friendly#2019-02-0712:20wilkerluciobut I can see that going wrong in some cases, did you got in one?#2019-02-0712:23liesbethWhere it goes wrong in my case is that there is for example a query Chat in my schema. The eql [{:pep-link.chat [:pep-link.chat/id]}] becomes query { chat { id } }, but I need query { Chat { id } }#2019-02-0712:24liesbethI don’t think that it’s going wrong (in the sense that the code is doing something else than was your intention), but my schema conforms to another standard where the root queries and mutations are written with a capital#2019-02-0712:25liesbethI’ve checked out pathom but I don’t see directly where I should make the change. Should I be looking at node->graphql?#2019-02-0712:29wilkerlucio@liesbeth thats a different problem I think, just to clarify, the GraphQL query would be something like: query { Chat(id: "...") { id name } }, is that correct?#2019-02-0713:04liesbethYes, that would be it#2019-02-0713:08wilkerluciocool, for this case you should also take a look at ident-map thing, that's the recommened way to run those via pathom, so in pathom this would become: [{[:service.chat/id "..."] [:service.chat/id :service.chat/name]}]#2019-02-0713:42liesbethThanks, got it hooked up!#2019-02-0712:32wilkerluciobut I think I see the issue, this is the problematic line: https://github.com/wilkerlucio/pathom/blob/a23c3e708932223823fa9ac97d2b4c7fdbdd8f58/src/com/wsscode/pathom/connect/graphql.cljc#L369#2019-02-0712:32wilkerluciobecause it uses a local version that doesn't allow for replacing the ::pg/js-name fn#2019-02-0713:05liesbeth@wilkerlucio Thanks! I’ll take a look at that šŸ™‚#2019-02-0713:07wilkerlucio@liesbeth please let me know how that goes, I would love to help and make the support more broad, since you actually have an use case lets get it fixed, I can take a revamp at the munging part tonight, if you can please list all the cases were the current transformation is failing, so far its about capitized entry points, but I guess you may find more#2019-02-0713:25liesbeth@wilkerlucio This is the only problem that I’ve encountered so far, but I’ve just started this morning šŸ˜‰#2019-02-0713:28wilkerlucio@liesbeth track via: https://github.com/wilkerlucio/pathom/issues/74#2019-02-0822:24hmaurer@wilkerlucio I demo’ed Pathom to a (older) friend yesterday (introduced him to demand-driven architectures; he wasn’t familiar with GraphQL either) and he raised the following question: how is this any different than the reason why SQL was created 40 years ago (allow clients to request data in a flexible languager), and why wouldn’t we then let clients run SQL queries directly? Now I had a few answers to that, going from security concerns to the fact that SQL is a complex language that is much harder to reason about, that there are benefits in limiting the power the client has, and that SQL doesn’t compose, but I don’t think I had a very good answer to his question.#2019-02-0822:24hmaurerDo you?#2019-02-1002:44eoliphantI’d say it’s apples and oranges. SQL is a great DSL for a very specific context (relational databases), with some rather conspicuous issues/limitations (string ’’orientationā€, etc)]. What pathom, and GraphQL (to a lesser degree) provide is a similar capability for a far more generic context#2019-02-1012:01hmaurer@eoliphant that was my initial reaction as well but after thinking about it I disagree that it’s apples and oranges. In Om.next David Nolen referred to the idea of ā€œmessagesā€ and ā€œinterpretationā€. A message is the bag of information (the request) that a client submits to the server, and ā€œinterpretationā€ is the process in which this bag of information is, well, interpreted to fulfill the client’s demands. You could start with very simple messages, such as ā€œgive me all usersā€, which wouldn’t be much different from a route in a REST API. You can then build more complex messages, such as those that Pathom interprets, which allow you to specify selections on fields, etc. There is, however, nothing stopping you from taking this further. Say you have a resolver to get a list of users filtered by name. You could imagine embedding a filtering DSL there, to filter by any arbitrary field with a semi-arbitrary condition composed of some primitives. Etc. If you went down that road you might end up with a language that, while not being SQL, gives similar powers to the client (almost arbitrary querying of your data)#2019-02-1019:33hmaurer@eoliphant does this make sense? šŸ™‚#2019-02-1019:46eoliphanthmm. fair point i think. But then is it just a matter of say pathom in particular perhaps being a ā€˜better’ apple? šŸ™‚ particularly in the clojure(script) world where it’s yet another expression of ā€˜doing stuff with data’ and there can be a pretty short ā€˜distance’ between EQL and however you’ve decided to implement resolvers when they are themselves ā€˜clojury’?#2019-02-1020:10hmaurer@eoliphant Yeah… I mean, I use Pathom, have used GraphQL and I get why, on an intuitive level, it is better to limit the power of such a language. But I don’t feel I have a very strong argument to answer people who bring up the above point on SQL#2019-02-1021:14aisamu> how is this any different than the reason why SQL was created 40 years ago (allow clients to request data in a flexible languager) It's not? It has better semantics for graphs but it's still a query language. But it's also a recursive data based query language, which makes it easier to break it apart for later recomposition. This is not as straightforward in SQL/GraphQL (even with fragments). > and why wouldn’t we then let clients run SQL queries directly? I'd say security and performance. You're the one that has to writing the executor/resolver/etc of the query "nodes", so you only give access to a blessed subset/family of queries. (e.g. a user can't poke at other user's data, a user can't perform an expensive search). Opt-in vs opt-out. You could argue that this would also be doable in a SQL system with some code, just a tad more annoying. I agree, but I'd expect a good amount of friction given the original design goals of the system/language.#2019-02-1021:17wilkerlucio@hmaurer hello šŸ™‚ ok, let me tell you how I see SQL in comparison with EQL, to start what is similar I think we can say that both are a way to describe information requests. Besides that there are some significant differences: 1. SQL outputs are tables, EQL outputs are trees. This is a very big difference, in SQL you can't describe nested structures, you can try faking it by adding extra columns but that goes off trace quickly, while EQL/GraphQL have in their core the ability to describe nested structures, that's one important reason why I think SQL is not suitable for user interfaces, imagine trying to write a single SQL query to describe a complex SPA data requirement with many deep levels of structure. 2. SQL is implementation specific, although some features of SQL have shared syntax, going from Postgres to Oracle, for example, can quickly break your queries, so its not in SQL nature are all to describe generic data structures, its about specific tables in a particular system, not for systems integration (which is what EQL / GraphQL does). 3. SQL has powerful aggregation tools, which can be good for reductions and processing, but are hardly so useful as an API, were you want more fined control of what are been exposed, if we go in this direction SQL and EQL are very different things with different goals 4. SQL is a text format (like GraphQL is too) while EQL is data-oriented (based on EDN). This allows easy of processing of a data format instead of having to deal with a new language/parser, this makes easy to write more generic operations since the requests are easy to manipulate.#2019-02-1021:18hmaurerThanks @wilkerlucio!#2019-02-1100:06Bjƶrn Ebbinghaus@wilkerlucio How does one use parameters when querying with an ident aka with a single input? There aren't any parameters in (:ast env) like usual. #2019-02-1100:07wilkerlucio@mroerni can you please share the query you are trying to run?#2019-02-1100:11Bjƶrn Ebbinghaus
edn
[({[:preference-list/by-slug "was-sollen-wir-mit-20-000eur-anfangen"]
   [:dbas.issue/slug {:preferences [{:position [:id :text :cost]}]}]}
  {:token "****"})]
Where my api looks like this:
clojure
(pc/defresolver preferences [{{{:keys [token]} :params} :ast} {slug :preference-list/by-slug}]
  {::pc/input #{:preference-list/by-slug}
   ::pc/output [:dbas.issue/slug {:preferences [:position [:id :text :cost]]}]}
 ....)
#2019-02-1100:12wilkerluciothat output seems wrong, this part [:position [:id :text :cost]]#2019-02-1100:13wilkerluciothat edn looks ok, altough there is another syntax that makes the params easier to find, like:#2019-02-1100:13wilkerlucio
[{([:preference-list/by-slug "was-sollen-wir-mit-20-000eur-anfangen"] {:token "****"})
  [:dbas.issue/slug {:preferences [{:position [:id :text :cost]}]}]}]
#2019-02-1100:14Bjƶrn EbbinghausThe query is made by fulcro with: (df/load this [:preference-list/by-slug slug] PreferenceList {:params {:token (:dbas.client/token connection)}})#2019-02-1100:14wilkerlucioah, that's fine, in the end they are equivalent#2019-02-1100:14wilkerluciobut you can't get that params in this level, because what's been processed is the :dbas.issue/slug#2019-02-1100:15wilkerluciowhat you could do is have a plugin or a custom ident reader to process this param and make it part of the env#2019-02-1100:16wilkerlucioexample plugin:
(def token-param-plugin
  {::p/wrap-read
   (fn [reader]
     (fn [env]
       (let [token (get-in env [:ast :params :])]
         (reader (cond-> env token (assoc : token))))))})
#2019-02-1100:16wilkerluciothis way no matter in what level the token is provided, anywhere deeper you will have the token#2019-02-1100:16Bjƶrn EbbinghausThat would actually be great. The token gets resolved to a user-id anyway in a bunch of mutations and resolvers.#2019-02-1100:17wilkerlucioanother suggestion, I'm seeing you using some simple keywords, that can go back quickly in pathom since all attributes live in a shared world#2019-02-1100:17wilkerlucioI suggest you use namespaced keywords everywhere#2019-02-1100:25Bjƶrn EbbinghausI will do that. Thank you very much. The plugin is exactly the right solution for me.#2019-02-1113:20Bjƶrn Ebbinghaus@wilkerlucio One more thing. If I want this param in my mutations as well, do I just have to use ::p/wrap-parser instead of ::p/wrap-read?#2019-02-1113:22wilkerlucio@mroerni you can use ::p/wrap-parser but that is a bit different since you get a full context and only once, you may want to look for ::p/wrap-mutate instead#2019-02-1113:22wilkerluciobut check some examples on how to use that (from pathom source) because mutations are a bit different because they return a map with an action, and what you want to wrap is the action#2019-02-1113:23Bjƶrn EbbinghausHow would I cancel the read, if the token is not present / invalid? Like a HTTP401.#2019-02-1113:25wilkerluciothrow an exception#2019-02-1301:31souenzzoNice thread about data/graph/app/domain/API/ https://news.ycombinator.com/item?id=19147742#2019-02-1316:16mssis there any way to force the reader in connect to return a sync value instead of a core.async channel? per the guidebook my parser config looks like:
{::p/env {::p/reader               [p/map-reader
                                        pc/parallel-reader
                                        pc/open-ident-reader
                                        p/env-placeholder-reader]
              ::p/placeholder-prefixes #{">"}}
             ::p/mutate pc/mutate
             ::p/plugins [(pc/connect-plugin {::pc/register (concat parser-mutations/mutations parser-queries/queries)})
                          p/error-handler-plugin
                          p/request-cache-plugin
                          p/trace-plugin]}))
#2019-02-1317:50wilkerlucio@mss can you tell more about what you want with that in the end? the parallel-parser will always return a channel given it uses a lot of core.async for coordination, if you want to run sync operations then you should use the regular parser, and use pc/reader2 instead of pc/parallel-reader#2019-02-1318:53mssjust playing around with the fulcro easy-server trying to glue it together to pathom. long term will be rolling my own server and the async mode is fine. just wasn’t clear in the dev guide how exactly to return a non core-async channel from the parser. all good, thanks!#2019-02-1319:13wilkerlucioyou can also just wrap the parser and make a core async block read, this way you make it sync#2019-02-1319:13wilkerlucioexample:
(def sync-parser [env tx]
  (clojure.core.async/<!! (real-parser env tx)))
#2019-02-1419:09mssa little bit confused about how resolvers are resolved under-the-hood. I have two resolvers that roughly look like:
(pc/defresolver user-data [env params]
  {::pc/input  #{:db/id}
   ::pc/output [{:user [:user/name :user/email]}]}
  ...)

(pc/defresolver current-user [env params]
  {::pc/output [{:user [:db/id :user/authenticated?]}]}
  {:user {:db/id               123
          :user/authenticated? true}})
I elided a lot of internals, but the rough idea is that the current-user resolver pulls a db/id, and that db/id is then fed into further resolvers. for whatever reason, though, I seem to be getting the following result:
(my-parser [{:user [:db/id :user/authenticated? :user/name :user/email]}]) =>

{:user {:db/id 123, :user/authenticated? true, :user/name :com.wsscode.pathom.core/not-found, :user/email :com.wsscode.pathom.core/not-found}}
so, two questions: 1) why are does the user-data resolver not seem to be getting invoked? what am I missing here? 2) is there a way to say which db/id should be used in a resolver? right now the input for the user-data resolver is just #{:db/id}. I assume there’s some method of distinguishing {:user [:db/id]} vs {:something-else [:db/id]}, or is there not? is that computed based on the output of the resolver?
#2019-02-1419:42hmaurer@mss try
(pc/defresolver user-data [env params]
  {::pc/input  #{:db/id}
   ::pc/output [:user/name :user/email]}
  ...)
#2019-02-1419:42hmaurerwill explain later why#2019-02-1419:53hmaurer@mss alright I am back. The short explanation is that you have an extra level of nesting in the output to your user-data resolver, as I pointed out above with the edited code example. According to your query you want to derive a :user/name and :user/email from a :db/id, but your user-data resolver allows you to derive a map from your a :db/id which has the :user/name and :user/email attributes#2019-02-1419:53hmaurerThe following query would work:
(my-parser [{:user [:db/id :user/authenticated? {:use [r:user/name :user/email]}]}])
but it’s clearly not what you want
#2019-02-1419:53hmaurerWhat you should do instead is have a flat resolver, e.g.
(pc/defresolver user-data [env params]
  {::pc/input  #{:db/id}
   ::pc/output [:user/name :user/email]}
  ...)
#2019-02-1419:54hmaurerAlso note that you should probably use :user/id for the ID, and not :db/id. If you use :db/id then you can’t differentiate between different kinds of IDs. Surely you can’t derive a :user/name from any ID, but only from a ID that is pointing to a user šŸ™‚#2019-02-1419:55hmaurer(that’s the answer to your second question)#2019-02-1419:57hmaurerSo to put it all together, I would have this:
(pc/defresolver user-data [env params]
  {::pc/input  #{:user/id}
   ::pc/output [:user/name :user/email]}
  ...)

(pc/defresolver current-user [env params]
  {::pc/output [{:current-user [:user/id :user/authenticated?]}]}
  {:current-user {:user/id               123
                  :user/authenticated? true}})
and as query:
(my-parser [{:current-user [:user/id :user/authenticated? :user/name :user/email]}])
#2019-02-1419:59mssthat makes perfect sense, knew I was hung up on something simple like that. really appreciate the help#2019-02-1419:59hmaurerno worries šŸ™‚ i had similar questions when I started learning Pathom#2019-02-1515:44kszabohas anyone tried to push resolver-level timing information (probably from taking data from the tracing plugin) to prometheus? How would that look like?#2019-02-1515:45wilkerlucio@thenonameguy we do exactly that at nubank, to prometheus, I have a plugin for it, let me get it for you šŸ™‚#2019-02-1515:48wilkerlucio#2019-02-1515:48wilkerlucio@thenonameguy I removed some specifics, but most of it is there ā˜ļø#2019-02-1515:49kszaboawesome! thanks#2019-02-1515:49kszaboI love code that I don’t have to write šŸ™‚#2019-02-1515:53kszaboone more question then: how do you debug a Fulcro application that is using Pathom Connect in stg/prod? It would be nice if you could just attach the Fulcro Inspector query tab / bundle it with the app#2019-02-1515:54kszabowe are using it over websockets so that could also be a problem. There have been times when I wanted to run arbitrary queries inside the app, but I don’t want to reimplement the query/tracing functionality myself / hook it up with Fulcro, unless it’s necessary#2019-02-1515:55wilkerlucioglad to help, about the run query, are you using workspaces?#2019-02-1515:56wilkerluciobecause if you are, you can easely hook a card to a parser: https://github.com/wilkerlucio/pathom-viz#workspaces-pathom-card#2019-02-1515:57kszaboI thought workspaces was mainly a development time tool#2019-02-1515:57kszabowe are not using it currently#2019-02-1515:58wilkerlucioit is, but we like to deploy it for operational things as well šŸ™‚#2019-02-1515:58wilkerluciothis is kind of a replacement for the old OgE (the pathom viz stuff)#2019-02-1515:58kszabohaha, okay, I will take a stab at integrating it then šŸ™‚#2019-02-1515:59kszabothanks again!#2019-02-1516:00wilkerlucio@thenonameguy another thing that might interest you is the one idea we apply using workspaces, in our setup here the user can switch (using the card toolbar) which remote he wants to hit, so the same card can be run against generative data, local running service or cloud running service, this gets a nice flexible way to try out each component#2019-02-1516:00wilkerluciothis setup is a bit more complicated, but I think its totally worth to make it, you do once and use it a lot#2019-02-1516:01kszaboohh, that sounds nice#2019-02-1516:01kszaboare you using clojure.spec’s as well to facilitate generating data?#2019-02-1516:01kszabowe started integrating https://github.com/gnl/ghostwheel here and there#2019-02-1516:02wilkerlucioyes, also pathom has support functions to generate data from components or queries#2019-02-1516:02kszaboand the combination of pathom + ghostwheel with autogenerated resolvers based on clojure.spec’s#2019-02-1516:02kszabowas on my mind#2019-02-1516:03wilkerlucioyou can make auto generated resoulvers if you have some consistent way to grab it, well, thats a big topic šŸ™‚#2019-02-1516:03wilkerluciohere is how you can generate from a query: https://cljdoc.org/d/com.wsscode/pathom/2.2.9/api/com.wsscode.pathom.gen#query-%3Eprops#2019-02-1516:04wilkerluciothis returns actual test.check generators, so you can compose with test.check stuff (supporting cool things like shrinking if you use on a test)#2019-02-1516:04wilkerluciothis is how you get a generator out: https://cljdoc.org/d/com.wsscode/pathom/2.2.9/api/com.wsscode.pathom.gen#query-props-generator#2019-02-1516:05wilkerluciothis is what a card looks like in our fulcro setup (for inspiration)#2019-02-1516:09kszabodamn, if only we had the story points for doing this šŸ˜„#2019-02-1516:10kszabothanks for showing this! If you could put this somewhere a bit more public (pathom wiki?) it would inspire many more people#2019-02-1521:21hmaurer@wilkerlucio out of curiosity how is your new Pathom explorer going?#2019-02-1521:24wilkerlucioI had a pretty busy week so it didn't moved much, but its on my top personal priority šŸ™‚#2019-02-1521:43hmaurershower thought: could an approach like Pathom be useful not just to fullfill the data needs of a client, but also within an application? It seems like the model of ā€œcomputational graphā€ where you can request pieces of information and derive them on the fly using a sequence of computations could be useful within a codebase :thinking_face:#2019-02-1613:10wilkerlucio@hmaurer I think about that a lot, it has some nice properties šŸ™‚#2019-02-1613:13wilkerlucioI imagine it can be work as a solution to the "business logic invocation problem", in other words a big controller that eliminates the need to write manual controllers, and instead delegate that to attribute relationships#2019-02-1613:15wilkerlucioI guess we can imagine a parallel with memory management, yes we could optimise the memory usage just for our programs and that was really fast, but at the cost of so many issues around bad memory management, can you imagine ideas of attribute relationship alleviating all this "bad input passing" that happens inside your code (specially around controllers code)?#2019-02-1614:23hmaurer@wilkerlucio yeah exactly. I am writing an application which involves passing a map of information down a sequence of ā€œstepsā€ to derive more information. Each step may use information from previous steps and may add information to the map. While I was writing it it struck me that it felt a lot like Pathom connect#2019-02-1614:28hmaurerI guess you might quickly get into the ā€œplanningā€ area of AIs though, concerned with getting to some objective by chaining actions in a certain way šŸ˜›#2019-02-1615:03wilkerlucioone other area I think it have interesting applications is in systems at large, if each service could expose a graph api, that can later be merged with other graph apis from other services, then the composed index could be exposed to everyone, making so every service knows how to get any info, so any information needed can be accessed directly by the service needing it#2019-02-1615:04wilkerluciohaving proper unique long names for the keywords can make this possible and efficient I think#2019-02-1615:05hmaurerhmm and so it would orchestrate ā€œhopingā€ between different services to get information?#2019-02-1615:09wilkerlucioyes, correct, since with the index the client knows all the required steps/paths to transition across the data#2019-02-1615:10wilkerlucioif we imagine a system with 100+ services, then it gets bad to each service to manually call each other to update index, but the system could have one service dedicated to do that#2019-02-1615:11wilkerluciothen when services deploy, they send the new updated index to C (lets say C is the service that knows the indexes)#2019-02-1615:12wilkerlucioso C knows about every service and their indexes, C could even make separated "resolver groups", picking and choosing which indexes to merge (so different parts of the system could request different resolvers views in some way)#2019-02-1615:12wilkerlucioso other services can request a index for a given resolver groups and use that to navigate though the system (both for request information and sending mutations)#2019-02-1615:13wilkerluciothat makes sense?#2019-02-1615:50hmaurer@wilkerlucio Yep it does. It’s an interesting approach for sure :thinking_face:#2019-02-1615:51wilkerluciothat's part of what motivates me to create that exploration graph we were talking about, so this network can be made visible#2019-02-1615:57hmaurerI look forward to seeing that šŸ™‚ I wonder how far you could push it as well… Like, say you have records in a database that can be rendered as PDF documents stored on S3. Could you somehow encode in a graph that these PDFs should be kept in sync (so you don’t have to wait for generation when you request the PDF).#2019-02-1615:57hmaurere.g. pre-compute parts of the graph#2019-02-1615:58hmaurerAlthough this might get into harder territory, if you want to ā€œreactā€ to changes in the data graph#2019-02-1615:59hmaurerBecause currently Pathom Connect works by running the computations (resolvers) on the fly#2019-02-1616:00wilkerluciosubscriptions could be interesting#2019-02-1616:00wilkerlucioand they could have a simple initial implementation, something like fulcro does for refreshing#2019-02-1616:00wilkerluciothat meaning, when some mtuatioin occurs, it tells the system to "refresh" some set of keyworks#2019-02-1616:00wilkerlucioand any subscription on that keywords get a new load#2019-02-1616:00hmaurerand that would also refresh all derived keywords#2019-02-1616:00wilkerlucioits differnet from reacting, but simpler to implement#2019-02-1616:00hmaurer(by derived I mean derived through resolvers, not the derive clojure thing)#2019-02-1616:01wilkerlucioyup, all the processes#2019-02-1702:03currentooranyone using pathom with nodeJS?#2019-02-1702:16wilkerlucio@currentoor should work same as running in the browser, are finding issues to setup?#2019-02-1702:17currentoorno just wondering if there was an example already written somewhere#2019-02-1702:17currentoorshouldn’t be too hard to figure out#2019-02-1702:17wilkerlucioyeah, you can hook on web framework like express#2019-02-1702:18wilkerlucioif you go for it, remember to return promise channels on the resolvers that do async calls (like http requests)#2019-02-1702:18wilkerluciopathom can leverage that on the coordination side šŸ˜‰#2019-02-1702:18currentoorsounds good simple_smile#2019-02-1702:19wilkerluciohttps://wilkerlucio.github.io/pathom/#_js_promises#2019-02-1702:22currentoor:thumbsup:#2019-02-1702:23currentooras part of our product we need on-site controllers to interact with equipment like bar code scanners and printers#2019-02-1702:23currentoorso i’m writing those as node servers on small linux devices#2019-02-1719:32currentoorI’m having troubled connecting pathom in-browser mutations to my fulcro application#2019-02-1719:32currentoor#2019-02-1719:33currentoorthat’s a gist of how i set it up#2019-02-1719:33currentoorbut when i trigger that remote mutation, like so in the Fulcro Inspect Query tab
[{(ucv.ui.receipt/print-receipt
   {:remote :rest,
    :order/id #uuid "4eb5af84-1611-4d79-8c3b-4acf919e1e82"}) 
  [*]}]
#2019-02-1719:35currentoorbut when i look at the indexes (shown in the gist) i see that this mutation is indeed in there#2019-02-1720:03currentoorIn the js console i see this error PathomRemote error: Error: Assert failed: Parse mutation attempted but no :mutate function supplied#2019-02-1721:13souenzzo@currentoor you need to provide ::p/mutate pc/mutate-async or something like https://wilkerlucio.github.io/pathom/#_mutations_setup#2019-02-1723:26currentoor@souenzzo thanks, i did that and it works from the repl#2019-02-1723:27currentoor
(pc/defresolver print-receipt [{:keys [app-atom ast]} {:keys [firm order]}]
  {#_#_::pc/params [:firm :order]
   ::pc/output [:receipt-printer/success]}
  (util/spy firm order)
  (go-catch
    (let [controller-api ""
          response       (async/<! (http/post controller-api
                                     {:body              (transit-clj->str
                                                           {:firm firm :order order})
                                      :with-credentials? false}))]
      (util/spy :pp (transit-str->clj (:body response))))))
#2019-02-1723:27currentoormy parser now looks like this
(defn rest-parser
  "Create a REST parser. Make sure you've required all nses that define rest resolvers. The given app-atom will be available
  to all resolvers in `env` as `:app-atom`."
  [app-atom]
  (p/parallel-parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/parallel-reader
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate-async
     ::p/plugins [(p/env-plugin {:app-atom app-atom})
                  (pc/connect-plugin {::pc/register (vec (vals @pathom-registry))})
                  p/error-handler-plugin
                  p/request-cache-plugin
                  (p/post-process-parser-plugin p/elide-not-found)]}))
#2019-02-1723:28currentoorand this works when i evaluate it in the repl
(comment
  (go-catch
    (util/spy (async/<! ((rest-parser) {}
                          '[(ucv.ui.receipt/print-receipt {})])))))
#2019-02-1723:28currentoorbut for some reason triggering it from fulcro does doesn’t work#2019-02-1723:28currentoorhowever, if i only change pc/defresolver to pc/defmutation then it works as expected#2019-02-1801:47currentoorAlso @souenzzo this approach doesn’t seem to have code reloading possible in dev, is that something that was working for you?#2019-02-1801:49currentoor@wilkerlucio perhaps if ::pc/register could be a function, then code reloading in dev would work?#2019-02-1801:50currentooror an alternative ::pc/register-fn?#2019-02-1814:28souenzzo@currentoor I never used #pathom on client. šŸ˜ž But hot-reload on pathon is a problem for me too. But i have a "complex" case where I generate the resolvers..#2019-02-1819:36wilkerlucio@currentoor @souenzzo hello guys, talking about refresh, its all about keeping the index updated. when we do the regular setup, all resolvers are send and the they are add to the index, by default the index will be created by the connect plugin under the hood, but you can send your own index atom, like this:
; create a index outside so you can maintain it
(defonce indexes (atom {}))

(def parser
  (p/parallel-parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/parallel-reader
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate-async
     ::p/plugins [(pc/connect-plugin {::pc/register my-resolvers
                                      ; send your index here
                                      ::pc/indexes indexes})
                  p/error-handler-plugin
                  p/trace-plugin]}))
Then if you all (swap! indexes pc/register some-new-resolver) it will be add to the index and take effect immediatly. Pathom doesn't have an opinion around this setup because there are too many ways to do it (and is often different for clj and cljs), but having your own index and writing some helpers around the core functions should give some way to handle it. Makes sense?
#2019-02-1819:37currentooryeah that totally makes sense, thank you#2019-02-1819:37currentooralso, is there any difference between using the parallel-parser vs the async-parser in the browser or node?
#2019-02-1819:38wilkerlucioyes#2019-02-1819:38wilkerlucioparallel parser can do requests in parallel, async supports async ops, but still serial#2019-02-1819:38wilkerluciounder the root they are very different šŸ™‚#2019-02-1819:39currentoori see, so the async parser is kind of like the request queue in fulcro?#2019-02-1819:39wilkerlucioit came in a time where parser was the only option, and js need async to do anything (like http requests)#2019-02-1819:39wilkerlucioso the async is just like the regular, but it supports core async channels for async ops#2019-02-1819:40wilkerluciobut it process in a serial way like the regular parser#2019-02-1819:40currentoorah i see, @tony.kay and I were wondering what the difference was#2019-02-1819:40currentoori assumed parallel only made sense in CLJ and async did requests in parallel#2019-02-1819:41wilkerlucionops, I suggest using parallel parser everywhere#2019-02-1819:41currentoorthanks for explaining#2019-02-1819:42wilkerluciono worries, one the topic, thats also why there are pc/async-reader2 and pc/parallel-reader#2019-02-1819:42wilkerluciothey work in very different ways, the paralle-reader needs to do way more work for orchestrating the parallel requests#2019-02-1819:42currentoori see#2019-02-2111:10Bjƶrn Ebbinghaus@wilkerlucio Is there a way to add aliases for inputs? I have resolvers which output {:foo/id 42}, so :foo/id would be the input for the foo-resolver But I would also like to query with an ident [:foo/by-id 42] Or should I call the resolver, instead of returning {:foo/id 42} Would it be a good idea, that the first resolver returns [:foo/by-id 42] which will then always be resolved? I hope you understand what I am thinking. šŸ™‚#2019-02-2111:17Bjƶrn EbbinghausSomething like this:
clojure
(pc/defresolver foo [_ {id :foo/by-id}]
  {::pc/input #{:foo/by-id}
   ::pc/output [:foo/id :foo/name]
  {:foo/id id :foo/name (str "Foo_" id}))

(pc/defresolver bars [_ _]
  {::pc/output [{:bars foo}]  <-- or like prim/get-query in fulcro
  {:bars [[:foo/by-id 1] [:foo/by-id 2]]}
such that
edn
[{:bars [:foo/name]}]

Results in:
[{:bars [{:foo/name "Foo_1"}
         {:foo/name "Foo_2"}]}]
#2019-02-2111:52mitchelkuijpersMight I recommend to change your idents to :foo/id. If you do that you could change your foo resolver to have #{:foo/id} as input. If you then change your bars resolver output to:
[{:bars [:foo/id]}]
And then if you return
{:bars [{:foo/id 1} {:foo/id 2}]}
Pathom will know how to resolve :foo/name
#2019-02-2112:18Bjƶrn EbbinghausBut :foo/id is a field, I am worried that it is confusing if it could also be an index. šŸ˜• What use are all these namespaced keywords, if I can not rely on what they are?#2019-02-2112:36souenzzoyou can do
(pc/defresolver bars [_ _]
 {::pc/output [{:bars [:foo/id]}]
 {:bars [{:foo/id 1} {:foo/id 2}]}
#2019-02-2112:38wilkerlucio@U4VT24ZM3 you can write a simple alias resolvers to convert one to the other (use (pc/alias-resolver2 :foo/id :foo/by-id)), but that said I suggest you just use the field name, its not problem at all, it makes simpler to understand (because in pathom anything can be made a connection, the idea of by-id been something special doesn't add much for you)#2019-02-2112:42Bjƶrn EbbinghausBut if [:foo/id 1] is the ident of {:foo/id 1} the resulting state would look like this: {:foo/id {1 {:foo/id 1}}} :foo/id denotes an index AND a field in an object in this index. Don't you think this is bad?#2019-02-2112:43mitchelkuijpersThis is only in the fulcro local db, that has nothing to do with pathom?#2019-02-2112:44mitchelkuijpers
{:foo/by-id {1 {:foo/id 1}}}
#2019-02-2112:45mitchelkuijpersThis is not such a huge deal to me, but you can choose to do either way, but we moved really fast to refactoring all by-id keywords to simple id keywords#2019-02-2113:00Bjƶrn EbbinghausI understand, that it won't be a huge deal, but I am feeling somewhat guilty about that, you know?#2019-02-2113:01mitchelkuijpersYeah I get it, but pathom sees it a bit different#2019-02-2113:01mitchelkuijpersIt looks at it like this: If i have an id i can get the following props for you#2019-02-2113:02mitchelkuijpersAnd you can go further you can also say if the map contains a id and something-else I can resolve even more for you#2019-02-2113:02mitchelkuijpersSo for you it says: if you have a foo/id I know how to get the foo/name#2019-02-2113:03mitchelkuijpersbut you could also write if you have: foo/first-name and foo/last-name I can resolve foo/full-name for you#2019-02-2113:04mitchelkuijpersso basically [:foo/id 1] is only syntax which it changes to {:foo/id 1} and then feeds to the pathom resolver#2019-02-2113:04mitchelkuijpersDoes this help?#2019-02-2113:13Bjƶrn EbbinghausI understand that, but I think that the ident-reader distorts the meaning of an ident (at least in the sense of fulcro), by making this change from [:foo/id 1] to {:foo/id 1} It looks at the ident as a part of the object, not a "reference" to the object.#2019-02-2113:26wilkerlucio@U4VT24ZM3 the ident is what it means, and "identifier", if we look at other places that have similar features in the clojure world, one is the datomic lookup refs, they look exactly like idents, and they point direcly to some attribute to enable further lookup. I see this fulcro idents as the same, the reason of by-id is most historical, [:foo/id 1] starts makes sense if you think that the attribute is like a primary key, and in pathom terms anything can be a key to something else, IMO when you start thinking on that direction than the by-id stops making sense and start seems just like some unnecessary overhead. makes sense?#2019-02-2114:01Bjƶrn EbbinghausTo look at it like a primary key makes sense, indeed.#2019-02-2113:41souenzzoHow can I print/capture the entity before it?
:cause "class clojure.lang.Keyword cannot be cast to class clojure.lang.IObj (clojure.lang.Keyword and clojure.lang.IObj are in unnamed module of loader 'app')"
 :via
 [{:type java.lang.ClassCastException
   :message "class clojure.lang.Keyword cannot be cast to class clojure.lang.IObj (clojure.lang.Keyword and clojure.lang.IObj are in unnamed module of loader 'app')"
   :at [clojure.core$with_meta__5405 invokeStatic "core.clj" 217]}]
 :trace
 [[clojure.core$with_meta__5405 invokeStatic "core.clj" 217]
  [clojure.core$with_meta__5405 invoke "core.clj" 217]
  [com.wsscode.pathom.core$elide_items invokeStatic "core.cljc" 219]
  [com.wsscode.pathom.core$elide_items invoke "core.cljc" 216]
#2019-02-2113:44wilkerluciowhat you mean entity before it?#2019-02-2113:58souenzzonot sure what i need to capture/debug#2019-02-2116:51wilkerlucioI dont understand what you want, its a debugging thing?#2019-02-2114:37pvillegas12I’m returning :fulcro.client.primitives/tempids from a pathom resolver but it is not being picked up by fulcro. I do see it in the response though :thinking_face:#2019-02-2115:36pvillegas12@wilkerlucio is there something pathom is doing to munge the tempids from fulcro? I’m getting a different looking datastructure after applying a defmutation in the backend#2019-02-2115:36pvillegas12Something like {:id #uuid "17ea4e29-41e8-4692-a521-825d87880fcb"} 5990139348123779}#2019-02-2115:48wilkerlucio@pvillegas12 check this https://wilkerlucio.github.io/pathom/#_mutation_join_globals#2019-02-2115:49wilkerluciowhat happens is that if you use some mutation response, only what your mutation response asked will be returned, and that can elide tempids, so global mutation joins is a feature to make that easier by always making it part of the output (when present)#2019-02-2115:51wilkerluciooh, but you are seeing the value changing? it should not change, if thats the case I suggest you take a look at the transfer layer (transit or whatever you are using), some data might be get compromised there (a common issue is not setting up fulcro tmpids with the transit, on the client or on the server)#2019-02-2115:52pvillegas12How do I do that? (set up fulcro tempids in transit)#2019-02-2116:45pvillegas12@wilkerlucio it appears that I needed some tweaking of my transit response function. However, the data I receive in that function does not have the temp ids correctly. Can pathom be changing the tempids?#2019-02-2116:49wilkerluciono, pathom doesn't touch your data in any way, unless you hit some weird bug, but I have not see it happening so far#2019-02-2116:50wilkerlucioyou need to check if the data is getting proper encoding, try logging at different places and see where its changing#2019-02-2116:51pvillegas12Currently I’m logging in the resolver before sending off the data#2019-02-2116:51pvillegas12where else?#2019-02-2116:51wilkerlucioyou can do in the result of the parser call itself#2019-02-2116:51wilkerluciobecause that's the last before encoding out (in most cases), so you can make sure there if there is any changing ocurring after you return it#2019-02-2116:52pvillegas12I’m looking at
(server/handle-api-request
                         parser
                         ;; this map is `env`. Put other defstate things in this map and they'll be
                         ;; added to the resolver/mutation env.
                         {:ring/request request}
                         (:transit-params request))]
#2019-02-2116:52pvillegas12but that call is already encoding it incorrectly#2019-02-2117:24pvillegas12@wilkerlucio @tony.kay found the culprit (p/post-process-parser-plugin p/elide-not-found)#2019-02-2117:24pvillegas12That pathom plugin was encoding tempids incorrectly#2019-02-2117:25pvillegas12Once I removed it, everything worked#2019-02-2117:26wilkerluciohumm, thats strange but a good catch, can you open an issue so I remember to debug it later?#2019-02-2117:59wilkerlucio@pvillegas12 I just got able to reproduce it in a consistent way, the issue is that (map? (fp/tempid)) returns true (which seems very weird to me), so the p/elide-not-found os navigating down on it#2019-02-2119:05wilkerlucio@pvillegas12 https://clojurians.slack.com/archives/C06MAR553/p1550775871007800 this version should fix the tempid issues with elide not found, @souenzzo this also bumps the keys to fix the issue you were having#2019-02-2119:29pvillegas12thanks @wilkerlucio! šŸ˜„#2019-02-2315:32pvillegas12Trying to make this query [{[:company/by-id 43078865576263770] [:db/id :company/name]} :com.wsscode.pathom/trace] and my resolver is defined as
(defresolver entity-resolver
  "Resolves query for an entity"
  [{:keys [conn tx] :as env} {:keys [company/by-id] :as input}]
  {::pc/input  #{:company/by-id}
   ::pc/output [:db/id :company/name]}
  (d/pull (d/db conn) '[*] by-id))
#2019-02-2315:33pvillegas12However, I’m not getting any data back from the UI / fulcro query#2019-02-2315:33pvillegas12(and I do have this data in the database, I am printing the result of (d/pull ...) and getting data back#2019-02-2317:01hmaurer@pvillegas12 did you solve this?#2019-02-2317:01pvillegas12Same error as in #fulcro#2019-02-2317:02pvillegas12šŸ˜„ thanks though#2019-02-2317:02hmaurerdid you try to run the query in Fulcro Inspect?#2019-02-2317:02pvillegas12yup, the datomic ids were getting parsed incorrectly on the way#2019-02-2317:02pvillegas12had various problems with the datomic ids#2019-02-2317:02hmaurerah ok!#2019-02-2401:20eoliphanthi, i’m trying to figure something out. I’m using Pathom with datomic. So, when I ask the parser to do something I’m passing in {:conn conn :db db} this generally gives me what I want. queries across resolvers have a ā€˜stable’ db, and mutations just do their thing. However, I have a problem when I’m doing mutation joins. Because, of course here [{(add-foo ...) [:foo/a :foo/b] I need my foo-by-id resolver to get an updated db in its env. Any suggestions?#2019-02-2402:42pvillegas12You could return the data structure you need within the mutation#2019-02-2402:42pvillegas12(instead of having another resolver do that)#2019-02-2403:45eoliphantwell the thing is, it’s not really part of the return value. the db ā€˜more rightly’ part of the env. and say my foo-by-id resolver shouldn’t care if part of a join, etc. BUt I think I’ve figured it out. I’ve created a plugin that sets a flag if there’s a mutation in a ::wrap-mutate, if it’s there, ::wrap-read will get the new db value, and reset the flag. feels a little clunky but it seems to work#2019-02-2402:43wilkerlucio@pvillegas12 hello, are you using the latest pathom? I did a release this week that did some changes to the elide-not-found plugin#2019-02-2402:43pvillegas12Yeah @wilkerlucio I am on the latest one, the problem was the encoding of large datomic ids#2019-02-2402:43pvillegas12I’m basically switching over to uuids for all entities#2019-02-2402:44wilkerlucioah, gotcha#2019-02-2402:45pvillegas12I remember you discussed this before @wilkerlucio do you have :entity/id per each entity in your system or do you have a shared :model/id that is a uuid?#2019-02-2402:46wilkerlucioone per entity, think of then as a first "filter", the name narrows the scope you have to search for it, but that can depend on how the db is modeled#2019-02-2402:46wilkerlucioin my use case I'm dealing with other micro services mostly, so they already have identifiers before me, and we use those#2019-02-2402:46pvillegas12the filtering just sold me 100% on uuids per entity type šŸ˜„#2019-02-2405:35eoliphanthi, i’m running into a weird problem, running at the REPL.#2019-02-2405:35eoliphanti’d noticed that the first time i’d call a newly instantiated parser#2019-02-2405:39eoliphantthere’s a bit of a lag (~ 5 secs), but subsequent calls are snappy. However, I noticed that if I don’t make a call for few mins, then try it again, I see that same lag. was wondering if there’s some sort of cache that might be timing out?#2019-02-2413:18pvillegas12might be datomic ion#2019-02-2417:02eoliphantthis stuff isn’t ā€˜ionized’ yet, just running in the repl and it’s making client api calls to the server. But I forgot to mention that I’d also wrapped the resolver bodies with (time) and have verfied that they aren’t the issue. Getting the same performance from them even when the calling the parser is ā€˜slow’#2019-02-2411:26wilkerlucio@eoliphant morning, had you tried tracing these slow requests to check what might be going on? 5s to run doesn't sound normal#2019-02-2416:59eoliphantyeah that my next question. Wasnt clear on how to get the tracing data while running from the the repl. I’ve done some simple checking like, wrapping the involved resolver bodies with (time) and have been able to verify that their performance is consistent#2019-02-2417:16eoliphantok nvm, just figured out how to add the :com.wsscode.pathom/trace key to the query. Well this is fun lol, seeing the same behavior even when that’s the only key. And the trace-results seem to indicate that what pathom is doing is pretty snappy hmmm… will keep digging#2019-02-2417:44eoliphantOk I think I was misreading it before. So @wilkerlucio if I’m reading this correctly, going from reset-loop to process-pending took about 5 secs?#2019-02-2417:45eoliphantand in a ā€˜good’ one that same jump is like 50 msecs
{:event "reset-loop",
                                       :duration 0,
                                       :start 4,
                                       :loop-keys [:com.wsscode.pathom/trace]}
                                      {:event "process-pending",
                                       :duration 0,
                                       :start 51,
                                       :provides #{[:fm.user/id #uuid"156e2937-5cf9-46fa-bbe6-8e9d9e1f3780"]},
                                       :merge-result? true}
#2019-02-2417:48wilkerlucio@eoliphant are you using fulcro inspect? you can get a much more detailed timeline view from there, its a lot of data to read by hand šŸ™‚#2019-02-2417:49wilkerluciobut that process pending is something that happen in side, so its the sum of the time some things happen before it, so not enough to know why it was slow#2019-02-2417:53eoliphantyeah, i have it but, don’t have the ui plumbing in place yet, probably will just do that lol#2019-02-2417:53eoliphantyeah, i’m learning as a go here lol#2019-02-2417:53eoliphantand I’ve now ā€˜traced’ it here#2019-02-2417:53eoliphant
:key :fm.user/first-name}
                                                              {:event "call-resolver",
                                                               :duration 5061,
                                                               :start 2,
                                                               :label :fm/user,
                                                               :input-data #:fm.user{:id #uuid"156e2937-5cf9-46fa-bbe6-8e9d9e1f3780"},
                                                               :sym :fm/user,
                                                               :key :fm.user/first-name}
#2019-02-2417:56wilkerlucioone thing that I see weird, why is :sym a keyword?#2019-02-2417:56wilkerlucioresolver names must be symbols#2019-02-2417:56eoliphantok now i’m just annoyed lol, it looks that that resolver code is doing something wonky. I’d just timed the datomic call but something else weird is going on#2019-02-2417:56eoliphantah#2019-02-2417:57eoliphantdammit lol, will fix that#2019-02-2417:57wilkerlucioIm curious, how/why you get a keyword as a resolver name?#2019-02-2417:58eoliphanti’m generating datomic schema, a bunch of resolvers, etc from this generic data model#2019-02-2417:58eoliphantjust a bug in that converter#2019-02-2417:58eoliphanti need to get them all spec’ed#2019-02-2417:59wilkerluciopathom has specs for most of the things, you can just do some (s/valid? (s/keys) x) check when you generated then, and pathom ns keywords will get validated#2019-02-2418:00eoliphantyeah, I saw that when I was poking around šŸ™‚ which is super helpful#2019-02-2418:02eoliphantside note, i wanted shoot you some doc PR’s But I’m getting the following error asciidoctor: WARNING: index.adoc: line 2691: include file not found: /Users/erichkoliphant/Dropbox/projects/pathom/src-docs/com/wsscode/pathom/book/graphql/fulcro_network/graphql_todo.cljs I created a feature branch off of develop in my fork, so it should be good in that respect#2019-02-2418:03wilkerluciohumm, strange, this example is not in my source either#2019-02-2418:03wilkerluciomaybe a bad link in the docs, we can remove this link#2019-02-2418:04wilkerlucioand improvements on docs are very welcome šŸ™‚#2019-02-2418:13eoliphantalso, did you see my note about needing to refresh the datomic db in my env in order to make mutation joins work? I’d been working all day and cobbled together a solution, but now with some sleep and coffee, I see some other potential issues. To fix the issue, instead of just passing in {:db db :conn conn} I now pass in an atom on its own key that looks like {:db db :conn conn :reset false}. Have a plugin (really great architecture on that BTW, super easy to use once you ā€˜get it’), that on wrap-mutate sets the :reset flag to true, and my wrap-reset if the flag is set, I swap in a new db value. and in either case, set the top-level :db key to the db from the atom. It works, but feels a little ah deselegante lol. My other concern is the ā€˜contract’. part of datomic’s value, especially for something like pathom/graphql is that I can run N disparate resolvers against the same DB state. In this case, while the mutation join now works fine. I’m not sure about the behavior when I send over say a mutation with a join, plus some number of queries.#2019-02-2418:14eoliphantok, cool, I’ll just yank that reference from the .adoc#2019-02-2512:30wilkerlucio@eoliphant its an interesting approach, and you are right to be concerned about the consistency across mutations and reads, at this moment pathom doesn't have an answer for this case, specially on the parallel parser since each root entry goes in parallel. that said I still need to see that happening, mutation responses are a much better place to request follow up info then follow-up reads, and if we consider that it might not be a problem, also, if multiple mutations need to coordinate, better transform then in a single mutation where you are free to coordinate as you need. so as I see we have ways around it as it is, the library could improve to have better support for it, but so far it seems to not be needed, makes sense?#2019-02-2514:54eoliphantHey @wilkerlucio, thanks for the confirmation re: my concerns. But I didn’t quite understand the > mutation responses are a much better place to request follow up info then follow-up reads by mutation response, did you mean a second ā€˜request’ based on the response from the first?
(parser {} [{(add-foo ..) [:foo/id]}])
(parser {} [{[:foo/id ...] [:foo/bar :foo/baz]}])
; - instead of 
(parser {} [{{(add-foo ..)[:foo/bar :foo/baz]}])
#2019-02-2514:54eoliphantalso shot you a quick doc pr#2019-02-2514:57souenzzo@eoliphant both
;; 1
[{(add-value) [:app/value]}]
;; 2
[(add-value)
   :app/value]
Are valid and will work But in 1 you say "do the mutation then do the query" In 2 you can "do the mutation and give me this query" In a parallel parser, 2 can query first and then do the mutation
#2019-02-2515:20eoliphantgotcha, so in my scenario though, I guess 2 is what I don’t want because I want the :app/value after the add-value?#2019-02-2517:25souenzzo@eoliphant I edited and corrected a detail in the query (removed [] from 2 ). No! 1 will always call :app/value after (add-value) 2 possibly call :app/value before/at the same time.#2019-02-2518:38eoliphantright right, I get that, so in most cases for me I need to use the join syntax, because I again, want to query against the thing the mutation just created/updated/etc#2019-02-2518:41souenzzo#fulcro also has the "mutation-merge", that you can use to merge your mutation results.#2019-02-2518:49eoliphantah sweet,I’d seen that haven’t used it though#2019-02-2519:06wilkerlucio@eoliphant I suggest you check then out, they are quite cool, and pathom deals nicely with then as well, in pathom case it uses the response data from the mutation as initial context, and you can query anything on top of it (as long as the mutation response has enough data to figure the rest) https://wilkerlucio.github.io/pathom/#_mutation_joins#2019-02-2600:46souenzzohttp://book.fulcrologic.com/#MutationJoins @eoliphant mutation-merge is now deprecared. That's the new (and clear) way#2019-02-2600:55eoliphantah sweet thx#2019-02-2601:04tony.kayAnyone seen infinite loops on pathom parallel parser? I have been using a parallel parser on this project for some time, and for larger responses it just started ā€œhangingā€ forever (but spinning GC/CPU). Dropping it to a normal parser fixes it (fortunately none of the resolvers returned channels yet)#2019-02-2601:13Joe LaneHi friends, when going through the pathom documentation for using the connect graphql remote, it uses the fake :demo-repos resolver. How would I specify a query with an argument? I’ve tried (df/load app {:event-by-id "c59cef71-f52e-45cf-a1c4-9960314b540e"} root/Root {:target [::root "singleton" :event-by-id]}) but I get an exception Uncaught Error: Assert failed: (or (util/ident? server-property-or-ident) (keyword? server-property-or-ident)). What am I missing here?#2019-02-2603:09wilkerlucio@tony.kay I just released yesterday one fix for a timeout situation on some error edge cases, not sure if it's the same thing, but please try 2.2.11 and let me know if improves, if doens't would be really helpful if you can get a minimal repro of it#2019-02-2603:28wilkerlucio@lanejo01 to send arguments for a query you can use a load with an ident, or use :params in the query options#2019-02-2603:29Joe LaneThanks! BTW, the documentation for the simple graphql complete solution is missing.#2019-02-2603:59wilkerlucio@lanejo01 sorry about that, going to fix, you can find the sources here https://github.com/wilkerlucio/pathom/blob/master/workspaces/src/com/wsscode/pathom/workspaces/graphql/simple_todo_demo.cljs#2019-02-2614:47Joe LaneThanks again!! This simple-todo-demo answers several other questions I had related to both fulcro and pathom.#2019-02-2616:11Joe LaneWould Pathom be open to a PR (or issue if the authors prefer) on the graphql-network function https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/fulcro/network.cljs#L190 I need to add auth headers to my simple graphql request and I dont know of any better place than a second arity or default argument. Am I missing something? Should this be done via middleware or some other mechanism?#2019-02-2616:25wilkerlucio@lanejo01 I'm open to the PR to add, we can have a arity 2 version of it that takes options map, that can include custom headers#2019-02-2616:35Joe Lane@wilkerlucio Great, I’ll be needing it shortly.#2019-02-2708:46Ahmed Hassan>The subquery for a join is in the :query of the environment.#2019-02-2708:47Ahmed Hassanwhat does this mean?#2019-02-2712:11wilkerlucio@ahmed1hsn in a query like: [{:foo [:bar]}], {:foo [:bar]} is the join, [:bar] is the sub-query of the join, and when doing processing, if you read the :query from the env var while processing that join, it will have the sub-query [:bar] there#2019-02-2712:11wilkerluciomakes sense?#2019-02-2714:10Ahmed HassanšŸ‘ #2019-03-0521:07lilactownare there any examples of using Pathom + GraphQL without Fulcro?#2019-03-0521:44thhellercheck the connect section. that should work without any fulcro#2019-03-0521:46lilactownit turns out this is all I need:
(my-ns
  (:require [com.wsscode.pathom.graphql :as pg]))

(pg/query->graphql [{:foo [:bar]}])
;; => "query { foo { bar } }"
#2019-03-0523:08lilactownhm. but it might not be possible to pass in parameters this way?#2019-03-0523:08lilactownhonestly the fact that apparently the Fulcro docs and the Pathom docs are interwoven are really frustrating#2019-03-0600:17souenzzo[{(:foo {:params "aa"}) [:boo]}]#2019-03-0600:36lilactownyeah I got confused by the idents bit in the docs#2019-03-0600:37lilactownbut it looks like I need to use idents for creating field aliases#2019-03-0600:37lilactownand the idents are complecting both parameters and alias; it's not very ergonomic when you want to read the data back#2019-03-0600:38lilactownI'm giving up on Pathom for now. just going to do string concat šŸ˜•#2019-03-2617:56souenzzo@eoliphant the resolver knows nothing about placeholder. if the query [{[:user/id 33] [:user/id :user/name]}] works, then the query [{[:user/id 33] [:user/id {:>/my-placeholder [:user/name]}]}] should work too.#2019-03-2618:30eoliphantHi @souenzzo, thanks, so I guess what I’m missing is, how do I ā€˜do/return something’ based on :>/my-placeholder? Do I create a plugin or something?#2019-03-2618:39souenzzoYou need to include the plugin in your parser somethign like this
(def parser
  (p/parallel-parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/all-parallel-readers
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate-async
     ::p/plugins [(pc/connect-plugin {::pc/register my-registers})
                  p/error-handler-plugin]}))
#2019-03-2618:40souenzzothen :>/my-placeholder can be any qualified keyword, that (#{">"} (namespace k)) return truth#2019-03-2618:43Alex H@eoliphant if you e.g. have a resolver that provides some :foo/* keys as well as a :user/id key, for example, And another resolver that provides :user/* given :user/id, you can either do a query along the lines of [:foo/a :foo/b :user/x :user/y], or you could e.g. use a placeholder like so, to build a bit of hierarchy: [:foo/a :foo/b {:>/user [:user/id :user/x :user/y]}]#2019-03-2618:44Alex Hthe only thing you need is to specify that ::p/placeholder-prefix bit that @souenzzo mentioned#2019-03-2618:44eoliphantright, I have the env-placeholder-reader, and prefix stuff setup in my parser.#2019-03-2618:44eoliphantah…. ok I think I misunderstood the proper usage.. So it’s like a ā€˜virtual’ key/node?#2019-03-2618:44Alex Hyep, it's just virtual hierachy#2019-03-2618:45eoliphantok ok got it#2019-03-2618:46eoliphantthanks a mil guys#2019-03-2700:11souenzzo@wilkerlucio can I get partial results from pathom? Kind of (partial-parser ctx [{[:user/id 1] [:user/name]}]) => {:result {:user/name "foo"} :ctx {..ctx..}} << here the parser will call the user-by-id, that return's {:user/name "foo" :user/active? true} then if I call (partial-parser ctx-from-the-return [{[:user/id 1] [:user/active?]}]) it will return {:user/active? true} without call user-by-id#2019-03-2700:13wilkerlucio@souenzzo yes: https://wilkerlucio.github.io/pathom/#_multiple_inputs#2019-03-2700:14wilkerlucioyou can send as param to any ident query, so your follow up read will be like: [{([:user/id 1] {:pathom/context ctx-from-the-return}) [:user/active?]}]#2019-03-2700:15wilkerluciothat gets easy to hook in Fulcro load in the :params setting doing an ident load#2019-03-2700:24souenzzoThis is nice. But I'm digging use lacinia to expose pathom as a graphql.#2019-03-2701:48wilkerlucio@souenzzo you can also try this: https://github.com/denisidoro/graffiti#2019-03-2712:15mitchelkuijpersThat looks very interesting!#2019-03-2712:33fatihictYeah, we might use this#2019-03-2712:33fatihictin the future#2019-03-2713:19souenzzonice! i will hack it.#2019-03-2701:49wilkerlucioand for gql compat, its quite easy to implement your own custom attribute to support this same feature, check the sources for the ident reader: https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/connect.cljc#L1367-L1375#2019-03-2713:44eoliphantHey @wilkerlucio, just got back around to looking at the pagination stuff again. One thing I’m trying to figure out is how to tie generic keys (per @mitchelkuijpers example)
[../has-next?
{:atlas-crm.pagination/items (prim/get-query Entity)}
           {:>/pagination (prim/get-query pagination/Pagination)}]
to the actual data that got resolved from say Entity. I guess in your example, the ā€œpaginated collectionā€ resolver is effectively wrapping the ā€œrealā€ collection so it’s more straightforward ?
#2019-03-2715:09mitchelkuijpersDo you have the placeholder plugin enabled in your parser?#2019-03-2715:10mitchelkuijpersThis is my pagination query if that helps:
[:atlas-crm.pagination/has-next-page?
           :atlas-crm.pagination/has-previous-page?
           :atlas-crm.pagination/size
           {:atlas-crm.pagination/items [:atlas-crm.pagination.items/cursor]}
#2019-03-2718:39eoliphantyes I have the placeholder stuff setup. Ok, but what I’m missing I think is how I can, for lack of a better term, ā€œlook downā€ (presumably from some pagination resolver?) at the stuff pointed to/resolved by the ..pagination/items in order to figure out the size, etc.#2019-04-0100:42hmaurer@wilkerlucio I was reading the draft for clojure.spec alpha2 this morning and it seems like it could integrate fantastically well with Pathom#2019-04-0100:42hmaurer:thinking_face:#2019-04-0115:27wilkerlucio@hmaurer yes, I do seem a lot of similarities around the selection part, really wonder how that will end up, selection is already an integral part of resolvers šŸ™‚#2019-04-0116:16hmaurer@wilkerlucio yep exactly, although Pathom’s selection operates on any map, whereas a selection with spec.alpha2 operates on a particular schema it seems#2019-04-0116:17hmaurerbut it still seems that something can be done using Pathom in combination with spec.alpha2#2019-04-0219:26lauritzshI watched the scaling talk and got really excited about Pathom. I am trying to wrap my head around it but I don't understand how it all works. The setup I am trying to achieve: having Fulcro talking with Pathom, persisting state in local storage/indexeddb. For persistence I might just use https://github.com/alandipert/storage-atom but datahike and konserve might scale better(?) TBD#2019-04-0219:27lauritzshIs it possible get some hints on how to achieve this? Google results are very limited, there are videos but it's hard to find exactly the info I need to get started and understanding it all#2019-04-0300:05wilkerluciohello @mail228, welcome, most of what you need should be present in the pathom documentation, did you give it a try? for the parser, there is not a lot special about the storages you mentioned, if you use local storage its simpler because its sync, but pathom also supports async so its also fine if you wanna use indexeddb or something like it#2019-04-0306:44lauritzsh@wilkerlucio Thanks, I skimmed the Connect chapter (3) and I think I got the parser up and running. I could make queries and have them resolve correctly. The problem I can't figure is how to keep Pathom resolvers and parser local and have Fulcro use that for querying. The template generates a server (which makes sense for most cases), but I haven't been able to it just client-side#2019-04-0308:19Bjƶrn EbbinghausHey, maybe you can use code I have written for a current project. Default Fulcro http-remote: https://github.com/hhucn/decidotron/blob/master/src/main/decidotron/client.cljs#L65 Server-side middleware for the api.. https://github.com/hhucn/decidotron/blob/master/src/main/decidotron/server_components/middleware.clj#L33 Pathom API + konserve: https://github.com/hhucn/decidotron/blob/master/src/main/decidotron/api.clj Ask if you want.#2019-04-0310:02lauritzshThanks, this seems to use a server for storing data, right? I need to just have everything on the client, in theory only have .cljs files in the project, since there is no /api endpoint to call#2019-04-0312:19wilkerlucio@mail228 I'm lacking a proper example for simple setup, but you can piggieback on this one: https://wilkerlucio.github.io/pathom/#_complete_graphql_connect_example#2019-04-0312:20wilkerluciothis is a graphql connect setup on client only, just remove the graphql parts and it should work, please let me know if that is enough to get a setup#2019-04-0312:25Bjƶrn Ebbinghaus@mail228 As I understand pathom, this ist fairly simple. You could nearly use my serversite code... and create a pathom-remote for fulcro https://github.com/hhucn/decidotron/blob/master/src/main/decidotron/remotes/dbas.cljs#L47 (the functions prefixed with dbas/ are actually returning core.async channels) and https://github.com/hhucn/decidotron/blob/master/src/main/decidotron/client.cljs#L68#2019-04-0312:27lauritzshThanks @wilkerlucio I will try it out when I get back from work šŸ™‚#2019-04-0312:27lauritzsh@U4VT24ZM3 Interesting, what does :dbas mean? I don't see it mentioned in the Pathom guide#2019-04-0312:28Bjƶrn EbbinghausIt is my own namespace. Has nothing to do with pathom#2019-04-0312:28lauritzshAh, is that a replacement for remote? I see you use (dbas in defmutation#2019-04-0312:28wilkerlucioyup#2019-04-0312:29wilkerluciojust like bjorn did as well, you create a pathom parser and use it as a fulcro remote#2019-04-0312:30lauritzshSo to understand, the name I give here https://github.com/hhucn/decidotron/blob/de7c9a11f035fe085049f55d5581424e0ef8a919/src/main/decidotron/client.cljs#L68 in this case :dbas is then available as (dbas in defmutation?#2019-04-0312:30lauritzshThen Fulcro understands this is possible to query and gets its data from that?#2019-04-0312:31Bjƶrn Ebbinghaus:dbas is the name of the remote, yes#2019-04-0312:33Bjƶrn Ebbinghaushttps://github.com/hhucn/decidotron/blob/de7c9a11f035fe085049f55d5581424e0ef8a919/src/main/decidotron/ui/components/login.cljs#L60 Here is the only place I use the dbas remote nowadays. (I don't use the remote, because I had to ditch a feature)#2019-04-0312:35lauritzshOkay, I think it's clicking now!#2019-04-0312:35lauritzshI start to see how it works#2019-04-0312:35lauritzshThanks so much both, I look forward to try later when I get home šŸ™‚#2019-04-0312:36wilkerlucioyou can also just use as a default remote, this way you don't have to specificy the remote name on the load calls#2019-04-0312:36wilkerlucioI usually give custom names only for extensions (when its something not related to the main app, or some special kinds of remotes like ones targetted for uploads, etc...)#2019-04-0312:37lauritzshHow do I specify a default remote?#2019-04-0312:38wilkerlucioin the code example I sent to you on graphql there is the full setup, basically just set as :networking (pathom-remote ...) or :networking {:remote (pathom-remote ...)}#2019-04-0312:38wilkerlucioon the fulcro side#2019-04-0312:39lauritzshAh so :remote is the default name?#2019-04-0312:40Bjƶrn EbbinghausYes, from the docs
`:networking` (optional). An instance of FulcroNetwork that will act as the default remote (named :remote). If
  you want to support multiple remotes, then this should be a map whose keys are the keyword names of the remotes
  and whose values are FulcroNetwork instances.
#2019-04-0312:49lauritzsh@U4VT24ZM3 How did you find that :thinking_face: I went here https://github.com/fulcrologic/fulcro clicked cljdoc, end up here https://cljdoc.org/d/fulcrologic/fulcro/2.8.8/doc/readme but I see no API docs. It does work on 2.8.0 though https://cljdoc.org/d/fulcrologic/fulcro/2.8.0/doc/readme#2019-04-0312:50Bjƶrn EbbinghausIt is in the docstring of the make-fulcro-client funktion.#2019-04-0312:51lauritzshAh makes sense, didn't think of checking source code, thanks šŸ™‚ I'll notify in #fulcro that namespaces docs are missing#2019-04-0315:44lauritzshLooks like it works, I can at least query from Fulcro Inspect, thanks for your patience @U4VT24ZM3 @wilkerlucio#2019-04-0315:45Bjƶrn EbbinghausYou are welcome#2019-04-0315:46Bjƶrn EbbinghausDon't forget to make use of the autocompletion feature. Click the grey circle in the top right corner of the query screen. I found it way too late.#2019-04-0315:52lauritzshohhh that's what it do lol I clicked it before I got it working and didn't understand#2019-04-0316:50wilkerlucioif you change the index (add resolvers/mutations) remember to click again on the ball to refresh it#2019-04-0321:51thhellerwhat is the state of the non-parallel parser? I'm trying to use it but somehow it works differently than the parallel one although I'm not doing anything async#2019-04-0321:51thhellerthe example doesn't in the readme doesn't seem to work for example#2019-04-0322:28wilkerlucio@thheller using the non-parallel parser requires you to use different readers#2019-04-0322:29wilkerlucioI suggest pc/reader2 instead of pc/parallel-reader, also need to use the regular mutate#2019-04-0322:31wilkerlucio@thheller try checking the example code right above this anchor, that should be a regular sync functional setup, please let me know if you find issues with it https://wilkerlucio.github.io/pathom/#_parallel_parser#2019-04-0322:31thhellerah I just commented out the parallel-reader#2019-04-0322:32wilkerluciobut that's the reader that does all the connect things, you may need it šŸ™‚#2019-04-0322:32wilkerluciobut use reader2 instead for the sync parser#2019-04-0322:33thhellerok thx, that works#2019-04-0322:33wilkerlucioin which part of the doc you tried to look for this information? so I may add it there, or a link or something#2019-04-0322:35thhellerall the examples used the parallel-parser so I thought the sync parser was maybe deprecated or so#2019-04-0411:04thhellerare there any utilities to create and run transactions directly against the pathom parser (without fulcro or so, directly from background tasks? I still absolutely hate the use of lists and symbols in transactions šŸ˜ž#2019-04-0411:05thheller
(defn create-request
  [{::env/keys [graph] :as ctx}]
  (let [tx-id
        `m/create

        query
        `[(~tx-id {})]

        order-id
        (-> (graph ctx query)
            (get tx-id)
            (get ::m/order-id))]

    ;; use order-id ...
    ))
#2019-04-0411:05thhellerthere has to be something better than this?#2019-04-0411:07thhellerI can easily abstract that myself and call something like (transact! env 'm/create {]) but was just wondering if there is something like that maybe?#2019-04-0411:40wilkerlucio@thheller nothing from standard lib, mainly because was never a use case for me, so better to create your own wrapper since you want a concise way to call it šŸ‘#2019-04-0411:41thhellerdo you only use the graph API from fulcro? I sort of started using it everything and except for the somewhat clunky API I really like that#2019-04-0412:07wilkerlucio@thheller what you mean use the graph api from fulcro?#2019-04-0412:07thhellerI mean fulcro like frontend that handles the db normalization#2019-04-0412:08wilkerlucioah, just read the only, currently yeah, but I keep feeling like it would be useful in more contexts#2019-04-0412:08thhellervs. a script of sorts that just one query without any db normalization or so#2019-04-0412:08wilkerlucioI was thinking making very trivial ways to create/run parsers#2019-04-0412:08wilkerlucioyou mean cluncky to call the parser?#2019-04-0412:09thhellerno calling the parser is easy. clunky in that I have to unwrap the results and stuff#2019-04-0412:09thhellerbut its easy enough to handle this with simple helper fns#2019-04-0412:10wilkerluciofor the parallel we need to do the channel read, for the sync one are you having to do some post processing too?#2019-04-0412:11thhellernot post processing, extracting the data (eg. get the tx-id in the example)#2019-04-0412:11thhellerits all fine#2019-04-0412:14wilkerluciocool, just out of curiosity, in which contexts are you using tha parser, like replacing regular logic in the program?#2019-04-0412:15thhellermostly using the parser to answer UI queries/mutations#2019-04-0412:15thhellerbut also for background tasks on the server directly#2019-04-0412:23thhellerI used to do straight db interop on the server but trying to do everything through the parser so its easier to trigger and monitor#2019-04-0412:25wilkerluciomakes sense, for those usages I do the same, any operations are first implemented in the parser and then used from it.#2019-04-0412:26wilkerluciobut I keep seeing these same patterns in my regular logic, I have some data, it maybe need a few steps to get the data I need, and I keep writing in the regular clojure style, which means I have to remember all the steps#2019-04-0412:27wilkerlucioso I wonder what would look like if the parser was a more integral part of my code, delegating all the data transitions to it#2019-04-0412:39thhelleryeah treating the parser sort of like a REPL endpoint#2019-04-0412:49thhellerI never tried this but is it possible to call transactions in some context?#2019-04-0412:49thheller
[{[:foo/ident 123]
  [(foo/do-something {})]}]
#2019-04-0412:52thhellerinstead of [(foo/do-something {:id 123})] and having the tx load the ident?#2019-04-0413:10wilkerlucioits technically valid, but I never tried doing it, I saw people doing similar things in GraphQL, I think its an interesting way to do things, could even operate on lists#2019-04-0413:10wilkerluciosomething like:
[{:customer/all-tags [(customer.tag/delete)]}]
#2019-04-0413:34wilkerlucio@thheller I did a quick run just for fun, it works already (kind of) šŸ™‚#2019-04-0413:34wilkerlucio
(pc/defresolver people-resolver [env _]
  {::pc/output [{:people [:person/name]}]}
  {:people [{:person/name "Tom"}
            {:person/name "Tal"}
            {:person/name "Lita"}]})

(pc/defmutation do-something [env input]
  {}
  (println "DO SOMETHING" input (p/entity env))
  {})
#2019-04-0413:34wilkerlucio
(parser {} [{:people [:person/name `(do-something)]}])

DO SOMETHING {} #:person{:name Tom}
DO SOMETHING {} #:person{:name Tal}
DO SOMETHING {} #:person{:name Lita}
=>
{:people [{:person/name "Tom", com.wsscode.pathom.playground/do-something {}}
          {:person/name "Tal", com.wsscode.pathom.playground/do-something {}}
          {:person/name "Lita", com.wsscode.pathom.playground/do-something {}}]}
#2019-04-0413:35wilkerluciothere is no auto-link between the mutation params and input, you need to grab it from the entity, but that can be easely automated#2019-04-0417:25thhellercool#2019-04-0510:22Ahmed Hassanp/join takes two arguments right? And I don't understand what should be first and second argument.#2019-04-0512:24wilkerluciop/join is quite a low level function at this stage, its still ok to use to use though, the point of join is moving the parsing context to a new one, so that usually happens when you have a join in your query, the arguments are the new entity data and the environment, the children AST will be extracted and a new instance of the parser will be called to process the sub-query#2019-04-0512:24wilkerluciothere are many details that evolved along this process, I suggest you try reading the sources and see if that makes sense, if you have any specific questions around it please let me know#2019-04-0801:09Chris O’DonnellI'm trying to write a plugin which adds an authorization header to graphql requests. This is what I have right now, but it doesn't seem to be adding the header as I expect it would. (I've checked that id-token-str has the expected value.) Any thoughts on what's going wrong?
(def auth-plugin
  {::p/wrap-parser
   (fn [parser]
     (fn [env tx]
       (let [{:keys [id-token-str]} @auth]
         (parser (cond-> env
                   (seq id-token-str)
                   (assoc-in [::p.http/headers :authorization] (str "Bearer " id-token-str)))
                 tx))))})

(def parser
  (p/parallel-parser
   {::p/env {::p/reader [p/map-reader
                         pc/parallel-reader
                         pc/open-ident-reader
                         p/env-placeholder-reader]
             ::p/placeholder-prefixes #{">"}
             ::p.http/driver p.http.fetch/request-async}
    ::p/mutate pc/mutate-async
    ::p/plugins [(pc/connect-plugin {::pc/indexes indexes})
                 auth-plugin
                 p/error-handler-plugin
                 p/request-cache-plugin
                 p/trace-plugin]}))
#2019-04-0809:40wilkerluciohello @codonnell, I don't see any graphql integration in your setup, usually the request for that is part of this integration, or just you just mean regular http calls?#2019-04-0810:32Chris O’DonnellWas trying to avoid pasting a whole bunch of code; I'll include a snippet with all of the relevant code.#2019-04-0810:33Chris O’Donnell@wilkerlucio I did mean graphql; I'm using the connect integration.#2019-04-0810:57wilkerlucio@codonnell gotcha, ok, so your setup looks correct, are you able to load the graphql schema without providing the auth? that's the only place I see that would not go though the plugin and so would miss the auth#2019-04-0810:59Chris O’DonnellOh derp, that is exactly the problem.#2019-04-0810:59Chris O’DonnellThanks šŸ˜„#2019-04-0811:03Chris O’DonnellNever mind, that was a problem. Still not seeing the authorization header get added when I move the schema fetching to after I add the token value.#2019-04-0811:04Chris O’DonnellUpdated init#2019-04-0811:15wilkerlucioI believe this new setup still doesn't send the auth, because it doesn't go though the parser, so you have to add the headaer in the my-gql when sending to load index#2019-04-0811:15wilkerlucioI did check the code, the env is send though from the parser to the gql, so I believe it will work once there#2019-04-0811:16wilkerlucioanother important thing to notice is that the specific namespaces are important on the connect integration#2019-04-0811:16wilkerlucioso since your prefix is ::pcg/prefix "little-gift-list.type", every attribute must start with that#2019-04-0811:17wilkerlucioso in the GiftList for example, instead of :query [::gift/id ::gift/name you probably want :query [:ittle-gift-list.type.gift/id :ittle-gift-list.type.gift/name ...#2019-04-0811:18wilkerlucioah, nevermind, I just realised you did proper alias for those, nice šŸ™‚#2019-04-0811:19wilkerlucioprobably still an issue with the load-index#2019-04-0811:24Chris O’DonnellAh of course, that makes sense. I replaced (pcg/load-index my-gql indexes) with (pcg/load-index (assoc-in my-gql [::p.http/headers :Authorization] (str "Bearer " (-> auth/auth deref :id-token-str)))) and can confirm that the header is added. Thanks for the help!#2019-04-0823:32Chris O’DonnellIs there a way to coerce values that come back in graphql responses? For example, I have uuids and timestamps which are encoded as json strings that I'd love to convert to uuid/datetime types before transacting them into my app db.#2019-04-0900:17Chris O’DonnellIt seems reasonable to me to accomplish something like this by walking the entity returned alongside the ast as a post-processing step, applying the coercion based on the key. Is there a good place to hook into the parsing process to accomplish this? I wrote a little inspector plugin that logs the env after each reader call has finished and found it was called twice with my query results (and twice with a pathom/trace key):
(def inspect-plugin
  {::p/wrap-read
   (fn [reader]
     (fn [env]
       (let [ret (reader env)]
         (js/console.log {:env env :ret ret})
         ret)))})
Not sure if that plugin is doing something funky, but it would be nice to inject a coercion function in a place where it just gets called once.
#2019-04-0913:28wilkerlucio@codonnell hello, that sounds a good extension point, currently there is already a parsing process that walks the graphql response to do things like munging and placeholder processing, but currently this part is not extensible#2019-04-0913:30wilkerlucioyou can see it here: https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/connect/graphql2.cljc#L321-L330#2019-04-0913:31wilkerluciocan you open an issue so I can track this idea to add support for graphql inner parser extensions?#2019-04-0913:39Chris O’DonnellWill do šŸ‘#2019-04-1020:48souenzzo[{(foo {}) [:bar]}] cant return {foo [{:bar 1} {:bar 2}]}. Pathom always suppose that the return of a mutation is a map. Is it intentional?#2019-04-1020:49wilkerlucio@souenzzo yup, that's by design, you see that everywhere, inputs must be maps, outputs must be maps, params must be maps, this is all intentional to force things to always compose (allowing extra things) and forces everything to be labelled#2019-04-1020:58souenzzofoo isn't a label? [{(:foo {}) [:bar]}] can return {:foo [{:bar 1} {:bar 2}]}#2019-04-1021:01wilkerlucioyes, but your response data also must be labeled#2019-04-1021:01wilkerlucioeg: {:person/id 123}, in this case 123 is labeled as :person/id, if it was just 123, no label#2019-04-1021:02wilkerlucioits about context independence, makes sense?
#2019-04-1021:03wilkerluciospecially if you consider mutation joins, if the mutation return wasn't a map it couldn't be extended#2019-04-1021:07souenzzoIt's a problem with #fulcro. I my component
(fp/defsc Chat [_ _]
  {:query [:chat/id
           {:chat/msg (fp/get-query Msg)}]
   :ident [:chat/id :chat/id]})
Then I have a mutation [(load-older {:msg/id older-msg-id})] It is called with (fm/returning Msg) This mutation need to be (df/prepend-to [:chat/id id :chat/msg]) So it generate [{(load-older {}) [:msg/id ...]}]
#2019-04-1021:09souenzzoWhen I do [{(load-older {}) [{:older [:msg/id ...]}]}], but this way, I can't merge correctly in fulcro state#2019-04-1021:10wilkerlucioare you using mutations to load data? any reason to don't use fulcro df/load?#2019-04-1021:17souenzzoATM I'm using df/load But once it will be called by "many" places, I thought that wrap it in a mutation makes sense. It also have some effects: set ui/loading? true, if returns empty, set final? true...#2019-04-1021:18wilkerluciofulcro load use the markers, I really suggest you keep using load, you can have wraps around it, but using mutations for data fetching goes off the semantics of the system... but anway, since you are adding multiple items to a list, there is no default helper in fulcro that will save you, when you load a resource its expected it directly maps to whatever component you are loading, but the modal doesn't have anything pre-made for specific injections (add this N items to a list), for those you will need a post-mutation anyway#2019-04-1021:20souenzzoI will try to use df/load again#2019-04-1021:20wilkerluciofor pagination I have some set of helpers that do what you want, but they require post-mutations to be involved#2019-04-1021:21wilkerlucio
(fm/defmutation merge-page-items [{:keys  [abrams.page/items]
                                   ::keys [page-key]}]
  (action [env]
    (db.h/swap-entity! env update-in [page-key :abrams.page/items] #(into items %))))

(defn load-next-page [this {::keys [page-key load-config]}]
  (let [ref           (fp/get-ident this)
        state         (-> (fp/get-reconciler this) fp/app-state)
        cursor        (-> (fp/props this) page-key :abrams.page/next-cursor)
        current-items (get-in @state (conj ref page-key :abrams.page/items))
        load          (merge {:query                [{ref [(-> (fp/get-query this)
                                                               (fp/focus-query [page-key])
                                                               first
                                                               (list {:abrams.page/cursor cursor}))]}]
                              :post-mutation        `merge-page-items
                              :post-mutation-params {:abrams.page/items current-items
                                                     ::page-key         page-key}}
                             load-config)]
    (fp/transact! this [(list 'fulcro/load load)])))
#2019-04-1113:52souenzzocom.wsscode.fulcro.db-helpers is on workspace dir. It not public (i think that can be) But I'm having progress. tnks#2019-04-1114:41wilkerluciothat db-helpers are internal on our project to be honest, but most of it is already ported in incubator šŸ˜‰#2019-04-1021:21wilkerluciosomething like this ^^^^#2019-04-1021:22wilkerlucioin my case all paginations use the :abrams.page/items standard, this way I can write generic pagination components that work across many different resources#2019-04-1118:12rschmuklerHave any of the developers of pathom considered making it so that the defresolver macro uses a constant symbol for the resolve function? In the context of repl driven development, it'd be nice to re-evaluate that macro without then having to also redefine the registry, and potentially restart a system (in the case of component or integrant). Is there much of a downside? If not, would you guys accept a PR making the change?#2019-04-1119:29wilkerluciothe initial implementations used to do something in this direction, they relied on multi-methods for dispatching, the issue is that the setup is much harder (because you have to hook many other things). Having a simple map really cleaned up a lot of things, also improves isolation since now the full resolver is contained in that map (without extenal deps, like having it "installed" in the multi-method)#2019-04-1119:30wilkerlucioI gotta move now, but I can tell you how to setup with multi-methods, the code still available and will be supported forever (although I like to encourage not doing so, because of the problems described, but still an option)#2019-04-1121:16mitchelkuijpersIf you create function during development which keeps creating a new parser then you can just redefine the macro and it will update#2019-04-1121:16mitchelkuijpersThat is what we do#2019-04-1121:17mitchelkuijpersSo we have something like this:
(defn parser [env query]
  (if utils/dev?
    ((create-parser) env query)
    (parser-inst env query)))
#2019-04-1121:22rschmuklerThanks for the replies! Sorry I didn't see them initially. The recreation of the parser is an interesting angle. I'll explore down that path and see where it gets me.#2019-04-1121:37rschmukler@U060GQK8U your solution worked quite nicely! Thanks for the suggestion.#2019-04-1123:33mitchelkuijpersNice! :thumbsup:#2019-04-1201:36wilkerluciore-creating parsers is all fine for dev, but please don't do that in a per request basis in prod, a single parser is better in that scenario#2019-04-1121:24rschmuklerAnother question: Is there any way to resolve the same key with different parameters as different names? For example [(:math/half-of {:n 4} (:math/half-of {:n 6})] results in {:math/half-of 2} with the resolver seemingly not even called the second time. I noticed that there's the ::pg/alias concept when porting EDN queries to GQL, but is there anything for handling it w/o using graphql?#2019-04-1201:36wilkerlucioyou can do it using: [(:math/half-of {:n 4 :pathom/as :another-name} (:math/half-of {:n 6})]#2019-04-1201:36wilkerluciobut there is a caveat here#2019-04-1201:37wilkerluciopathom does cache all resolver calls by default#2019-04-1201:37wilkerlucioand currently there is a problem that it doesn't consider the params in this caching, this will change in the future, but for now you have to remove cache for that to work, otherwise both calls will get the same value (because only the first will call the resolver, second will hit the cache)#2019-04-1303:51wilkerlucio#2019-04-1417:15Bjƶrn EbbinghausIs there any way this visualization is coming to fulcro inspect?#2019-04-1420:40wilkerlucioit is there already :)#2019-04-1518:01Bjƶrn EbbinghausOh.. It could be, that I never restarted Chrome... Handy, that my laptop crashed an hour ago.#2019-04-1518:09wilkerlucioyeah, also Chrome doesn't update the extensions immediately, so it might take days until it does the update, but you can manually ask for it to get the latest version#2019-04-1518:13Bjƶrn EbbinghausHm. But now I get an exception, when I try to load the index. šŸ˜ž
Mon Apr 15 20:12:10 CEST 2019 [worker-2] ERROR - POST /api
java.lang.RuntimeException: java.lang.Exception: Not supported: class com.wsscode.pathom.trace$fn__15355
	at com.cognitect.transit.impl.WriterFactory$1.write(WriterFactory.java:65)
	at cognitect.transit$write.invokeStatic(transit.clj:167)
	at cognitect.transit$write.invoke(transit.clj:164)
...
#2019-04-1518:21wilkerluciothe issue is with Transit in this case, because there is some things in the index that transit can't encode by default, in this case a fn, there are 2 things you can do to go around: 1. be more specific about the output index to avoid sending fns, since you write the resolver for the index explorer you can control it there 2. configure the default transit handler so it doesn't break on not supported things I like option 2 better because it goes around all encoding issues that might happen#2019-04-1518:24wilkerlucioto do 2 you can follow the example from Fulcro Inspect itself: https://github.com/fulcrologic/fulcro-inspect/blob/303d5fd99685b34bfb122a59c532b6a5781654bc/src/client/fulcro/inspect/remote/transit.cljs#L19#2019-04-1719:12mitchelkuijpersAwesome tip thnx! :thumbsup:#2019-04-1515:04thheller@wilkerlucio would you be open to making all the specs in .pathom.connect conditional? something like (when SPECS ...) and with a goog-define or so?#2019-04-1515:05thhellercurrently pathom adds up to almost as much code as cljs.core in :advanced and the specs aren't used there as far as I can tell#2019-04-1515:05thhellerso the output is huge for no reason šŸ˜›#2019-04-1515:56wilkerlucio@thheller oh, that's a fair point, I think I'm ok adding that, but just in the opposite direction, we can make a flag to disable the specs#2019-04-1515:56wilkerluciomaybe we could even try to make some generic name for it, I guess you may want to remove the ones from eql as well#2019-04-1515:56wilkerluciomaybe something like CLOJURE_SPECS_DISABLE#2019-04-1515:57thhellerthe best option would moving the specs into a separate namespace#2019-04-1515:58thhellerbut that gets a bit icky with namespaced keywords#2019-04-1516:37wilkerlucioyeah, my main motivation for putting it togheter was always about convenience of reduced number of requires, but considering this size problem in cljs I may change my policy#2019-04-1519:02rschmuklerHey @wilkerlucio, regarding your comment yesterday on :pathom/as for aliasing responses, it seems that that doesn’t work with ::pc/bulk? true - is that intended? If not, should I open an issue?#2019-04-1519:03wilkerlucioyou mean ::pc/batch? true?#2019-04-1519:03rschmuklerYes, sorry!#2019-04-1519:03rschmuklerIt seemingly only applies to the first item in the batched response#2019-04-1519:04wilkerluciointeresting, yeah, please open an issue, that's not intended#2019-04-1519:05rschmuklerOkay! Also, thanks for your response on https://github.com/wilkerlucio/pathom/pull/88 - still waiting for a bit more clarification but happy to help if I can!#2019-04-1519:14wilkerluciocomment replied šŸ˜‰#2019-04-1519:24rschmuklerHey @wilkerlucio if you have a second to discuss that PR, let me know - I’m absolutely happy to change it, I just wan’t to make sure I understand correctly.#2019-04-1519:25wilkerlucioI guess I did replied already, I'm missing some other comment?#2019-04-1519:27rschmuklerNo, sorry, I just figured it might be faster to discuss on Slack. Specifically, won’t just adding it to the cache-batch function only make param caching work for batch requests? Also, won’t that mean that if it was resolved in a non-batch fashion earlier, it won’t be used if its later resolved in a batch? ie. Won’t one of them use params, and one of them not?#2019-04-1519:28rschmukler(And I’m totally happy to implement it that way, I just want to make sure that you’re okay with that, and that I’m not misunderstanding this code so that I can help more in the future šŸ™‚ )#2019-04-1519:29wilkerlucioI mean, I replied on Github, did you saw that there? I'm confused on which point of the discussion you are#2019-04-1519:30rschmuklerI saw! And again, sorry, I must be missing something#2019-04-1519:31rschmuklerhttps://github.com/wilkerlucio/pathom/blob/dc0ff004ae887e94ae2085df027c43a148b352de/src/com/wsscode/pathom/connect.cljc#L950#2019-04-1519:31rschmuklerConsider that line - isn’t that necessary such that the parameters get considered (and not collide) so that the inner body is even run a second time?#2019-04-1519:32wilkerluciook, I may understand now, so yes, for the cached-async calls you need to send p#2019-04-1519:32wilkerluciois that what you pointing to?#2019-04-1519:32rschmuklerYes#2019-04-1519:33wilkerluciocool, that's right, those calls still need p, my suggestion will be to remove from cache-batch and its calls#2019-04-1519:33wilkerluciosounds good?#2019-04-1519:33rschmuklerOkay! Will do!#2019-04-1519:48rschmuklerReady for re-review when you are! Thanks again for the responses.#2019-04-1520:20wilkerluciothanks! I did a quick look and seems good, I wanna do some manual checks, probably some time during this week and then we can get it in, thanks for the work!#2019-04-1519:41wilkerlucio@thheller do you wanna do it? or maybe file an issue so we keep tracking it#2019-04-1519:46thheller@wilkerlucio too swamped with other stuff currently. happy to do it when I find some time though. https://github.com/wilkerlucio/pathom/issues/90#2019-04-1705:12Ahmed HassanI want to communicate with Postgresql for fetching and posting data from Pathom in Fulcro application, are there any examples and code for help to get a started? I'm using Honeysql for queries. (Additional question) What migration library do you use for SQL DBs?#2019-04-1706:13claudiudid you check out this video https://www.youtube.com/watch?v=gbrdnSsUerI&amp;list=PLVi9lDx-4C_Rwb8LUwW4AdjAu-39PHgEE ? There is also https://walkable.gitlab.io/ case you want to experiment#2019-04-1713:22hjrnunesHello, I'm trying to get this example going, but I'm puzzled by the response. Why am I getting channels out? Can anyone shed any light on this? Thanks! Query:
(parser {} [{:t/data [:t/id :t/text :t/test_train :t/label]}])
Response:
#:t{:data #:com.wsscode.pathom.parser{:provides #{:t/data},
                                      :response-stream #object[clojure.core.async.impl.channels.ManyToManyChannel
                                                               0x70c6abd9
                                                               "
Parser impl:
(def data [{:t/id         1
            :t/text       "asadad"
            :t/test_train "train"
            :t/label      "a"}
           {:t/id         2
            :t/text       "assdgsd vvdad"
            :t/test_train "train"
            :t/label      "b"}
           {:t/id         3
            :t/text       "asadadw kjonjo"
            :t/test_train "test"
            :t/label      "c"}])

(defn mk-resolver []
  {::pc/sym     (symbol "my-ns" "my-resolver")
   ::pc/output  [{:t/data [:t/id :t/text :t/test_train :t/label]}]
   ::pc/resolve (fn [_ _] {:t/data data})
   })

(defn mk-index []
  (-> {} (pc/register (mk-resolver))))

(def parser
  (p/parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/parallel-reader
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/plugins [(pc/connect-plugin {::pc/indexes (atom (mk-index))})
                  p/error-handler-plugin
                  p/request-cache-plugin
                  p/trace-plugin]}))
#2019-04-1713:24hjrnunesI'm using the map version of the resolver because I intend to generate them dynamically later on. Is this even a good idea at all, btw?#2019-04-1714:07wilkerlucio@hjrnunes hello šŸ™‚ the issue in your code is that you are using a serial parser combined with the pc/parallel-reader, those are not compatible, pc/parallel-reader requires the parallel-parser, if you just want serial parser you can replace the pc/parallel-reader with pc/reader2#2019-04-1714:17hjrnunes@wilkerlucio Ah, I see! Makes sense#2019-04-1714:17hjrnunesthanks#2019-04-1714:18hjrnunesIs there a big advantage in going parallel for simple stuff?#2019-04-1714:18wilkerluciothe parallel (as the name says) can run things in parallel, it really depends on how your data is distribuited#2019-04-1714:19wilkerluciosome system will be more parallelizable than others, as you grow it usually gets good things, but in some cases the serial can be faster (when there is no room for parallelism)#2019-04-1714:19hjrnunesI see#2019-04-1714:20wilkerlucioalso, the mk-resolver helper you created, there is one in pathom already, pc/resolver or pc/defresolver#2019-04-1714:20hjrnunesDoes it allow me to register any symbol?#2019-04-1714:20wilkerlucioyup, whatever you send on the map will be the one it picks#2019-04-1714:21wilkerluciobut keep in mind that resolver names doesn't matter much, they just need to don't collide with others#2019-04-1714:21wilkerlucioyou rarely reference resolvers by symbol#2019-04-1714:21hjrnunesSo is there any problem in creating resolvers dynamically at all?#2019-04-1714:22wilkerlucionot at all, the graphql integration does a lot of that#2019-04-1714:22hjrnunesCool!#2019-04-1714:23hjrnunesI'm thinking about using EQL for a very simple data language. I don't actually need the tree aspect of the data, but I could use the expressiveness of the query parameters. Something like a poor man's Walkable, but without all the SQL bells and whistles, and without joins#2019-04-1714:23hjrnunesLike to query one single table#2019-04-1714:24hjrnunesHowever, I do need global query params, so to speak#2019-04-1714:24hjrnunesWalkable does the :table/all join to get a bunch of rows, and I can apply :limit to that join#2019-04-1714:25hjrnunesis that the best way?#2019-04-1714:26hjrnunesProbably wasn't very clear. EQL params have to be applied to either a property, ident or join, right?#2019-04-1714:26hjrnunesthere's no way to apply them to the whole query itself?#2019-04-1714:55hjrnunesAlso, in the resolver, how do I get the parameters? Is it supposed to be from the query in env?#2019-04-1714:59wilkerlucio@hjrnunes yes, you get from env (-> env :ast :params), about the globally aplication, that can be done with plugins, you can write a plugin that captures the params on read and make it part of env, doing that will propagate that inforamtion to every subchildren, still you can't do at query level, but can be at something that queries everything else#2019-04-1714:59wilkerlucioto do query level you can use query meta-data if you want#2019-04-1714:59wilkerluciothat's a bit different but works as well#2019-04-1715:00hjrnuneshmm I see. An explicit join is probably clearer. Thanks!#2019-04-1715:03hjrnunesis (-> env :ast) supposed to contain the portion of the AST relevant for the resolver, or do I get-in with the path?#2019-04-1715:06hjrnunesI got it#2019-04-1715:07hjrnunes@wilkerlucio Thanks for your help and work on Pathom and Walkable. All in all, really useful and impressive work!#2019-04-1715:16wilkerluciothanks, but I can't take the one on Walkable, that one you need to thank @U0E2YV1UZ šŸ™‚#2019-04-1715:58hjrnunes@wilkerlucio can I manipulate ::pc/processing-sequence to filter out entities? Basically, I want to apply a limit to the results per property value. [{:ents [:e/prop1 e:/prop2 (e/prop3 {:limit 2})]}] would return at most 2 rows for each distinct value of :e/prop3. I'm aware this can be done at the join level, but I like the expressiveness of doing it like this, because I'll be having a typical limit at the join level too.#2019-04-1715:59wilkerluciomessing with processing-sequence is a bad idea, thats an internal thing#2019-04-1715:59wilkerluciomaybe what you want can be better served as a plugin#2019-04-1715:59wilkerlucioso you can parse params and limit results across the whole parsing#2019-04-1716:00hjrnunesyeah, so i'd collect any :limit params at the property level, stick them on the env, and then filter out when resolving :ents?#2019-04-1716:13wilkerlucio@hjrnunes hum, I would be careful about using limit on env, sounds weird that a limit in a parent would affect some joins many levels down, usually that is a thing per join#2019-04-1716:14wilkerlucioyou still can do it, but you would be complecting parts of the query in a way that's not obvious#2019-04-1716:14wilkerluciowith a plugin you can implement the limit once and it will work anywhere the user adds the param (in joins returning lists)#2019-04-1716:14wilkerluciothats a quite easy plugin to write#2019-04-1716:18wilkerlucio@hjrnunes
(def limit-plugin
  {::p/wrap-read
   (fn [reader]
     (fn [env]
       (if-let [limit (-> env :ast :params ::limit)] ; use qualified keyword so it doesn't clash with other people things
         ; let-chan only required to support async/parallel parsers
         (let-chan [res (reader env)]
           (if (sequential? res)
             (vec (take limit res))
             res))
         (reader env))))})
#2019-04-1716:51hjrnunes@wilkerlucio I'll give it a go! Thanks again!#2019-04-1718:28mitchelkuijpersI was wondering something about pathom, since you can use placeholders in queries. Do people actually use it instead of the default fulcro parser?#2019-04-1718:48wilkerlucio@mitchelkuijpers I did some tests around it, the problem is that pathom is too slow to work as the client parser, client parser needs to be really fast (and it tries to optimize on that, for example, it doens't even convert the query to AST, parse the query directly to reduce overhead), pathom on the other side is designed for flexibility, so a lot of wrappers and injection points that add overhead#2019-04-1718:48mitchelkuijpersAh that clears it up, then I wont waste any time on it. I was just curious#2019-04-1718:50wilkerlucio@mitchelkuijpers but if you need for other purposes (maybe want to simulate fulcro reader on the server), pathom has that setup ready too: https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/map_db.cljc#2019-04-1718:51mitchelkuijpersAh nice, No was just wondering since we sometimes use placeholder queries with pathom. Since it makes structuring some UI stuff a bit easier#2019-04-1718:52wilkerlucioyeah, the good part is that the client parser doesn't need to know about that, in the end it turns in regular fulcro links šŸ™‚#2019-04-1718:53mitchelkuijpersYeah doesn't really matter for Fulcro luckily#2019-04-1718:54mitchelkuijpersBtw I tried out the index browser today but it seems to fail on:
::pc/transform pc/transform-batch-resolver
#2019-04-1718:55mitchelkuijpersWith this error:#2019-04-1718:55mitchelkuijpers
[INFO] [talledLocalContainer] Caused by: java.lang.Exception: Not supported: class com.wsscode.pathom.connect$batch_resolver$fn__36671                                                               
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.marshal(AbstractEmitter.java:194) [?:?]                                                                                
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.JsonEmitter.emitMap(JsonEmitter.java:171) [?:?]                                                                                        
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.emitMap(AbstractEmitter.java:85) [?:?]                                                                                 
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.marshal(AbstractEmitter.java:184) [?:?]                                                                                
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.JsonEmitter.emitMap(JsonEmitter.java:171) [?:?]                                                                                        
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.emitMap(AbstractEmitter.java:85) [?:?]                                                                                 
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.marshal(AbstractEmitter.java:184) [?:?]                                                                                
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.JsonEmitter.emitMap(JsonEmitter.java:171) [?:?]                                                                                        
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.emitMap(AbstractEmitter.java:85) [?:?]                                                                                 
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.marshal(AbstractEmitter.java:184) [?:?]                                                                                
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.JsonEmitter.emitMap(JsonEmitter.java:171) [?:?]                                                                                        
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.emitMap(AbstractEmitter.java:85) [?:?]                                                                                 
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.marshal(AbstractEmitter.java:184) [?:?]                                                                                
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.emitArray(AbstractEmitter.java:97) [?:?]                                                                               
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.marshal(AbstractEmitter.java:182) [?:?]                                                                                
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.emitTagged(AbstractEmitter.java:49) [?:?]                                                                             
#2019-04-1718:56mitchelkuijpersBut now that I am reading it is probably because it tries to convert a fn to transit#2019-04-1719:09wilkerlucioyup#2019-04-1719:10wilkerlucio@mitchelkuijpers check this https://clojurians.slack.com/archives/C87NB2CFN/p1555352482010500?thread_ts=1555127499.001300&amp;cid=C87NB2CFN#2019-04-2001:24rutledgepaulvhi! i recently became aware of pathom and really appreciate the way resolvers compose and let the consumer worry less about how the data is connected / how to navigate to their desired data from what they already have. That said, I’m not a fulcro user and am only looking for read support, not mutations and some of the other things. I have some ideas of what the api for a more minimal / standalone resolver library might look like but i’m struggling with how to go about pathfinding between provided input and desired output. Is there any literature / tips on how to implement that? I’m familiar with basic graph theory but perhaps not how best to model this. could someone who is familiar with it discuss how pathom achieves this with me? I’m guessing a lot of it has to do with what the pathom index looks like.#2019-04-2001:31wilkerluciohello @rutledgepaulv, welcome šŸ™‚ the path finding of pathom was a pragmatic implementation, I didn't based on any formal literature (although I had past experience writing Dijkstra algorithms), and you guessed right about the index, it has a lot to do with that. There is a section in docs explaining the indexes, had you checked that? https://wilkerlucio.github.io/pathom/#_understanding_the_indexes#2019-04-2001:31wilkerlucioabout the finding, the heart of it you can find this section in connect: https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/connect.cljc#L619-L659#2019-04-2001:33rutledgepaulvthanks! No i haven’t looked at those yet. I started trying to write some of the pathfinding stuff and realized I didn’t know how it was working lol#2019-04-2001:33wilkerlucioand that said, I like to just clarify that you don't need to use Fulcro to use Pathom, its a stand alone library, a parser in the end is a function, you can create it and call in any clojure or clojurescript program, no need to integrate with any visual thing. but if really wanna get on the core and write your own I'll be happy to talk about impl details šŸ™‚#2019-04-2001:33rutledgepaulvthanks. I wasn’t sure how connected (ha) it was to fulcro#2019-04-2001:33rutledgepaulvalso just curious about how to implement such a thing though#2019-04-2001:34wilkerlucioFulcro and Pathom have a very good synergy, but are independent things#2019-04-2001:34rutledgepaulvok#2019-04-2001:35wilkerlucioif you can understand that snippet I sent, its pretty much all the work on the path finding, based on the index it goes from an input set to some output attribute, it returns all possible paths#2019-04-2001:35rutledgepaulvperfect#2019-04-2001:36rutledgepaulvI’ll look through those and perhaps follow up with more questions#2019-04-2023:32Chris O’Donnell@wilkerlucio Are you open to a PR for https://github.com/wilkerlucio/pathom/issues/86? I'd love to see a solution for it, and I'm happy to do some legwork to make it happen. If you are, it would be great to discuss what you'd like a solution to look like at a high level so I'm not working in a completely different direction. If not, no worries. šŸ™‚#2019-04-2023:50wilkerluciosure, I would be happy to take a PR for that, I was just looking into the code to see what it takes, and maybe its not so hard, the core would be around this line: https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/connect/graphql2.cljc#L395#2019-04-2023:51wilkerlucioI'm imaging that instead of calling parser-item directly, it could check if there is a ::parser-item defined on the config (so it works in a per-graph impl model), and if a parser-item is provided, it uses that instead of the default one#2019-04-2023:51wilkerlucioso the user can implement an entire custom parser to handle the inner parsing, makes sense?#2019-04-2023:54Chris O’DonnellYep, makes sense.#2019-04-2100:19Chris O’DonnellUnless I'm missing something, this is a very small change. Do you have any thoughts on how to test it (if you feel that is necessary)?#2019-04-2100:55Chris O’DonnellPR up at https://github.com/wilkerlucio/pathom/pull/91. Happy to add tests or documentation at your discretion.#2019-04-2100:56Chris O’DonnellSmoke tested it using a personal project.#2019-04-2101:36wilkerlucionice! yeah, I think its a small change, can you also update docs to include this?#2019-04-2101:44Chris O’DonnellWorking on it. šŸ™‚ Never used asciidoc before, so progress is a little slow.#2019-04-2102:01wilkerluciothanks, asciidoc is a bit different but very fun once you get it, also quite powerful šŸ™‚#2019-04-2102:02wilkerluciotip: if you want to compile the book, you can run make book (you need to have ruby and install the bundle)#2019-04-2102:35Chris O’DonnellThanks, that was helpful. šŸ‘#2019-04-2102:52Chris O’DonnellGood god, the emacs adoc mode makes indenting code in-buffer impossible.#2019-04-2116:38wilkerlucio[com.wsscode.pathom "2.2.14"] is out! This release includes: - Consider params in resolver cache key. thanks to @rschmukler - Allow custom inner graphql parser. thanks to @codonnell - In source docs for diplomat specs. thanks to @souenzzo Also add a minor helper p/params to read params from env (shortcut for: (-> env :ast :params (or {}))).#2019-04-2203:05souenzzoHow about that optional "syntax" to define mutations/resolvers https://gist.github.com/souenzzo/a0f9624e7dd20589ca9ad8905334b7cc PS: ::pc/kind isn't a good name#2019-04-2212:09wilkerlucio@souenzzo I'm not seeing much gain with this syntax, whats about the pc/defresolver or pc/resolver that doens't work for you? you can also use a plain map:
(def my-resolver
  {::pc/input #{...}
   ::pc/output [...]
   ::pc/resolve (fn [env input] ...)}
#2019-04-2217:39souenzzo@wilkerlucio it's "hard" do test At the end, it's just a function with some metadata Testing shoud be (my-resolver {:data 33} {:param 22}) => {:result 11}. ATM to test I always need to repeat (let [resolve-fn (::pc/resolve my-resolve)] ...) I'm already using that var-metadata version internally. Not sure if pathom should care about that. Maybe pathom should KISS and always expect maps..#2019-04-2218:46wilkerlucio@souenzzo if calling is the problem, you can make a helper for that: (defn call-resolver [resolver] ...), that seems the easiest path me, what you think?#2019-04-2219:00souenzzoI'm using var metadata and I'm very happy with it. I will see if it scale or not, then I feedback I think that many libs like #clara can also use var metadata too.#2019-04-2219:12wilkerlucio@souenzzo pathom used to suggest multimethods, the problem is when you want to break things apart, in many cases I found myself wanting to break down indexes (use just parts of it), specially for testing, and multimethods make close to impossible to separate once they are glued, that's the main motivation for the new design, keep things separated and make it easy to join#2019-04-2219:12wilkerlucioyou can just def vector with all the resolvers and that's composition for pathom#2019-04-2602:43Chris O’Donnell@wilkerlucio I'd like to build a graphql query which has a vector of enums in its params. It's my understanding that symbols in EQL queries are translated into enums in graphql queries, but that doesn't seem to be the case if they're in a vector inside the params. For example,
[{(gql/build-thing {:columns [id name description]}) [:clientMutationId]}]
is compiled into
mutation {
  build-thing(columns: ["id","name","description"]) {
    clientMutationId
  }
}
when (I think) it should be compiled into
mutation {
  build-thing(columns: [id, name, description]) {
    clientMutationId
  }
}
Is my interpretation of the query->graphql translation correct? If so, I'm happy to submit a little PR that takes care of this. If not, is there some other way I don't see to get a vector of enums into graphql params?
#2019-04-2602:45wilkerlucio@codonnell yes, your interpretation is correct, if it was symbols it should be using the ENUM way out, glad to take a PR to fix#2019-04-2603:07Chris O’Donnellhttps://github.com/wilkerlucio/pathom/pull/92 whenever you get a chance.#2019-04-2914:54kszaboare there any plans to support global resolver resolution in Connect mutations? I would love the automatically get :current-user/name for logging purposes, and I already wrote a resolver for it#2019-04-2915:23kszaboIf ::pc/input would work as with plain resolvers, it would be great#2019-04-2915:26souenzzoi think that resolvers should receive something like ::pc/query#2019-04-2915:31kszabo
{::pc/input  [:utc-time/now :current-user/email]
 ::pc/params [:arg1 :arg2]
 ::pc/output [:model/id]}
#2019-04-2915:31kszabothis reads nicely to me#2019-04-2915:32kszaboit would also signal that this has the same semantics as a resolver input#2019-04-2918:05wilkerlucio@thenonameguy what you mean supports global resolution? if you jsut want to run a subquery, you can call the parser from the inside, something like:#2019-04-2918:05wilkerlucio
(pc/defmutation do-thing [{:keys [parser] :as env} _]
  {::pc/sym 'user/do-thing}
  (let [user-name (parser env [:current-user/name])]
    ...))
#2019-04-2918:05wilkerlucio(remember to read the channel in case you are using async or parallel parsers)#2019-04-2918:06wilkerlucio@souenzzo what data you would like available as ::pc/query?#2019-04-3001:47kszabo@wilkerlucio I essentially wanted to write ā€˜This mutation requires these parameters explicitly and these resolver’s outputs implicitly, Connect resolve the latter for me please’ your way is the manual version of that. I’m okay with that solution as well but I think somewhere read that in Connect there is only a fine line between resolvers with side-effects and explicit mutations, so I assumed ::pc/input would work (to fetch from env/call resolve automatically) the same.#2019-04-3001:51wilkerlucio@thenonameguy not really, but you can use mutation joins to request extra data, and you could write your own plugin that does that, I would suggest a different key (something like ::expose), if you add that key to your mutation it will stay there (the config is an open map, you can add anything you like)#2019-04-3001:52wilkerlucioso in the plugin code, extending with ::p/wrap-mutate you can pull that and execuse (using a parser call)#2019-04-3001:52wilkerluciomakes sense?#2019-04-3001:54kszaboyup, thanks!#2019-04-3001:54kszabosince you are already here#2019-04-3001:56kszabohave you considered how Pathom could make use of the nav and datafy protocols? REBL seems like the same graphy-domain-exploration-thingy that Pathom could be if it had larger reach#2019-04-3002:40wilkerlucio@thenonameguy yes, it could use it to expand data, I didn't got the chance to play with that yet, but I imagine that you could the current auto-complete features to extract context, and probably suggest possible paths, so you could keep navigating#2019-04-3012:58kszabohey @wilkerlucio, I tried to use Pathom Connect’s Index Explorer with transit+json encoding, and since the returned ::pc/indexes contain functions the encoding fails as by default transit doesn’t have a handler for fns. Would it make sense to update the example resolver code in the docs to remove these fns via postwalk?#2019-04-3012:59kszaboI assume they won’t be used#2019-04-3013:57kszabo
(pc/defresolver viz-index-resolver [env _]
  {::pc/input  #{:com.wsscode.pathom.viz.index-explorer/id}
   ::pc/output [:com.wsscode.pathom.viz.index-explorer/index]}
  {:com.wsscode.pathom.viz.index-explorer/index (walk/postwalk #(when-not (fn? %) %) (get env ::pc/indexes))})
#2019-04-3013:57kszabothis way the Index Explorer works (and is AWESOME! šŸŽ‰ )#2019-04-3015:55wilkerlucio@thenonameguy that's a concern outside pathom scope, for example, if you are using a local parser in CLJS, would be fine to send fns there (since there is no encoding needed), we may even inspect that fn, but I understand that its a common pitfall for users of the backend, a solution that I suggest is to setup transit default handlers, I think this is a good practice since after that nothing will break your encoding (and I believe that's what a lot of people would want)#2019-04-3016:00kszabosure, just a note would have been nice. As I think most people are going to use Fulcro with PC and I pulled some hairs out trying to figure out why it didn’t work. This is what I would need to patch: https://github.com/fulcrologic/fulcro/blob/develop/src/main/fulcro/websockets/transit_packer.cljc#2019-04-3016:02kszaboand with the current setup I just got a network disconnect after 30 seconds, without any error response/logging server-side (which I know is a Fulcro WS wrapping issue but it was frustrating).#2019-04-3018:16wilkerlucio@thenonameguy agreed better docs are needed since its a common pitfal#2019-05-0415:36geraldodevpathom isn't archived on https://clojurians-log.clojureverse.org/#2019-05-0514:23eoliphantanyone using pathom with fulcro’s form state stuff? I’m trying to figure out the best way to get dirty-fields into something useful for the server#2019-05-0912:22kszaboare there any ways to handle short circuiting outside of implementing these as resolvers? Let’s say I have :low-cost/x :mid-cost/x :high-cost/x which are all aliases to :domain/x. My clients generally want them to be in low-cost order (which isn’t necessarily the same order as what the network weights are). Is there a general purpose reader extension lurking around this idea where the client can request something like (:domain/x {::p/resolve-order [:low-cost/x :high-cost/x]) for instance?#2019-05-0912:56kszabooh, I see, I think ::pc/sort-plan is what I’m looking for, it’s just not at the attribute level#2019-05-0922:26souenzzoanyone else using fulcro+pathom "in anger"?#2019-05-1002:41wilkerlucio@souenzzo what "in anger" means?#2019-05-1004:06claudiuThinking of https://m.youtube.com/watch?v=8o01g6C7jWg&amp;t=1s#2019-05-1013:13souenzzoin production, with a team.#2019-05-1020:46wilkerlucioI'm šŸ™‚#2019-05-1021:26souenzzoI'm too šŸ™‚ But I'm curious about how many of us#2019-05-1014:51souenzzoA resolver should ALWAYS return everything that it's know? Can I consider it a bug? https://gist.github.com/souenzzo/c120fdc37311341bffd97036ec339766#2019-05-1014:55wilkerlucio@souenzzo no, it doesn't have to, the issue you see is because of caching#2019-05-1014:55wilkerluciothe first call will cache the result, and since from the first to the second the input didn't changed it will hit the cache#2019-05-1014:56wilkerlucioyou can set ::pc/cache? false (in the resolver config) to disable it, but then you need to manual cache yourself if you want avoid multiple calls to the same resolver#2019-05-1019:48souenzzoIt's good to be able to choose. I will add a comment here.#2019-05-1120:43eoliphantHi, I’m adding pathom to an existing datomic project where the entities are essentially ā€˜duck typed’, so just the internal :db/id, and external UUID :entity/id, and the code determines if you’re a person (or a duck) based on the presence of certian attributes. Trying to figure out the best way to handle lookup resolvers, where I want say a ā€˜person’ of entity/id XXX.#2019-05-1120:49hmaurer@eoliphant fetch the entity with id XXX, check if it’s a person, and return nothing if not?#2019-05-1121:03wilkerlucio@eoliphant think in terms of lookup range, having a more specific name gives you more information, its same as :id vs :customer/id#2019-05-1121:09hmaurerah sorry, I misunderstood your question 😬#2019-05-1121:10eoliphantyeah I was just trying to think of ways to make it work ā€˜as is’ šŸ™‚. From the pathom perspective, I think that might be inefficient right? If I have person, duck, foo, etc resolvers? if it keeps checking then discarding them. But, ah hell. it would still possibly work based on the fields you ask for right? If my person resolver takes :entity/id, but outputs :person/name …, pathom would find the right one if I ask for {:entity/id XXX [:person/name ...]}#2019-05-1121:11eoliphantbut yeah I can always add the ā€˜typing’ attributes#2019-05-1318:23kszabois there anyway to disable parallel key timeouts? My usecase is that I want a delay computation, but reuse the request cache, so I would like to call p/entity-attr! later. I put a lambda function with that call on a core.async channel to be processed later. If by that time the resolver has run I want to reuse it, otherwise calculate it on the spot, without duplicating the resolvers code. Sadly I get a key process timeout. What could I do differently?#2019-05-1318:29wilkerlucio@thenonameguy the parallel timeout is configurable, I'm not sure if you can disable it but you can make it a large enough number so it works, the default is 60s (just for context, this is kind of a last resource failing system, if nothing works this tries to guarantee the user will get some response after that time), you can change the value using environment data, you need to set :com.wsscode.pathom.parser/key-process-timeout 60000 (default value in this example)#2019-05-1318:31wilkerlucioabout reusing the cache, one trick you can do is also send the cache atom in the env, example: (let [cache (atom {})] (parser {::p/request-cache cache} [...]))#2019-05-1318:31wilkerluciothis way you can hold to the cache after the request is done#2019-05-1318:51kszaboThanks! why can’t I just get the existing (default) ::p/request-cache from the env in the mutation I’m firing off this async lambda and pass it back in? It should be a reference type, no need to put it an ā€˜initial’ atom myself, I think#2019-05-1319:23kszabowith the high timeout hack I could progress but p/entity-attr! says that it’s unable to resolve my attr, while the entity has all the necessary inputs to do so#2019-05-1319:23kszaboI’ll workaround this for now, I’ll revisit it by tmrw#2019-05-1319:23wilkerlucio@thenonameguy its just that you don't have access to the env at the output level, the request cache is created one per request and then thrown away, so altough you can access during in-processing, there is no external access point for it, makes sense?#2019-05-1319:24kszaboyes it makes sense, but I’m firing off this async stuff in a mutation, where env is in context#2019-05-1319:24kszabowith it the per-request cache AFAIK#2019-05-1319:24kszaboso I don’t need to put it in manually#2019-05-1319:24kszaboif my assumptions are correct#2019-05-1319:25wilkerlucioah, in this case you can pull it, I'm not sure exactly how you are trying to do it, but by the description it seems correct, one thing to remember is that the case is based on resolvers, so what you cache is not the entity data but the resolver results, knowing that may help you#2019-05-1319:25wilkerlucioanother thing, p/entity-attr! uses ::p/entity from the environment, so make sure that this data point has the inputs you need#2019-05-1319:28kszabo
Entity attributes #{:some/response} could not be realized
#:com.wsscode.pathom.core{:entity {:foo true, :java/inet-address #wp/inet "8.8.8.8"}, :path [[:java/inet-address #wp/inet "8.8.8.8"]], :missing-attributes #{:some/response}}
#2019-05-1319:28kszaboand :some/response only depends on :java/inet-address#2019-05-1319:28kszabo
{::pc/input  #{:java/inet-address}
   ::pc/output [:some/response]}
#2019-05-1319:28kszabolooks ok to me#2019-05-1319:29kszabo
(pc/defmutation async-maybe-resolve [env params]
  {}
  (async/go 
    (async/<! (async/timeout 1000))
    (some-side-effect (async/<! (p/entity-attr! env :some/response))))
  true)
#2019-05-1319:29kszabothe gist of what I’m trying to do#2019-05-1319:30kszabooh, imagine a timeout before the take#2019-05-1319:31wilkerlucioyeah, looks correct, I never tried from a mutation though, may be a bug, if you can get a minimum repro of the issue I can take a look#2019-05-1319:32kszabook, it’s getting pretty late here in Budapest, but I’ll try a bit more tmrw and post results šŸ™‚ thanks again for Pathom, it rocks#2019-05-1319:33wilkerluciohave a good rest#2019-05-2112:58kszabohttps://github.com/wilkerlucio/pathom/issues/29 @wilkerlucio do you have any ideas how this would work out? I’ve been toying with the idea of switchable micro-service/local resolvers#2019-05-2113:43wilkerlucio@thenonameguy the idea here is similar to how GraphQL integration works today, a custom resolver with dynamic output, but there are current a few complications to implement that properly, one of the challenges is computing intermediate dependencies when connecting multiple resolvers with dynamic output, that's a current issue with the GraphQL integration as well#2019-05-2220:23rschmukler@wilkerlucio I think I have a minimal reproducing case for https://github.com/wilkerlucio/pathom/issues/89 which I just added. I've poked around the source to see if I could fix it but I'm a bit out of my depth. Any help would be hugely appreciated.#2019-05-2315:45wilkerluciothanks!#2019-05-2822:36daniel.spanielI have a basic pathom 101 question that has me puzzled. Let's say I have an entity called 'thing' and I want to query for attributes of thing like thing/name and thing/age .. and usually pathom lets you pass in the query and that works fine BUT if the attributes are not in the pc/output then the query params are not used, so it seems like I have to list all the thing attributes in the pc/output value ?? is that true ? am I missing something .. so here is my resolver#2019-05-2822:36daniel.spaniel
(defresolver thing-resolver
  "Resolves query for a thing"
  [{:keys [conn] :as env} {:keys [thing/id] :as input}]
  {::pc/input  #{:thing/id}
   ::pc/output [:db/id :thing/id]}
  (d/pull (d/db conn) (::core/parent-query env) [:thing/id id]))
#2019-05-2822:37daniel.spanieland if I query for thing/name and or thing/age they will not be returned since they are not declared in the ::pc/output#2019-05-2822:37daniel.spanielthis seems odd, but maybe that is the way it is ?#2019-05-2901:19wilkerlucio@dansudol hello, so, your guess it has all the things is true, that's the connect index, attribute resolution is done in this layer, there is also the map reader layer, so in case you already had that loaded you may get it, but that can be an accident, its important to have the possibilities in the index (which comes from the inputs/outputs), this allows for introspection and smarter processing, you can read more about connect indexes at: https://wilkerlucio.github.io/pathom/#_understanding_the_indexes#2019-05-2906:17Ahmed HassanI'm trying to understand GraphQL and Pathom. What I don't understand is how does it compares with Lacinia, except that we can use it in ClojureScript too. Pathom seems superset of functionality that Lacinia provides.#2019-05-2910:13hmaurer@ahmed1hsn the biggest difference is that graphql/lacinia are entity-level, whereas Pathom is attribute-level#2019-05-2910:45Ahmed HassanWhat difference does it make?#2019-05-2911:54daniel.spanielthanks @wilkerlucio I think i get it .. the thing attributes have to be laid out by at least one ( in this case "get me the thing" resolver ) and now pathom/connnect knows the attributes of thing and can use that anywhere else you ask for things#2019-05-2914:33wilkerlucio@ahmed1hsn hello, both GraphQL and Pathom try to fix the same problem of structured requests, TBH is more fair to compare GraphQL and EQL, in the same way you have multiple GraphQL processors (Apollo, Lacinia, Relay...) you may have multiple EQL processor (although currently Pathom is the only impl of eql that I know of). You can find more details on how they compare in EQL docs: https://github.com/edn-query-language/eql#graphql-comparison#2019-05-2914:33wilkerlucioplease let me know if you need more clarification#2019-05-2914:34wilkerlucioalso is worth knowing that Pathom has some support for GraphQL integration (which can be confusing for new people trying to understand the relationship)#2019-05-3015:47souenzzoThere is a simpler way to do this
(->> [:foo
      {:bar [:var]}]
     eql/query->ast
     :children
     (mapv :dispatch-key))
;;=> [:foo :bar]
#2019-05-3017:10wilkerluciono, no helper for this specific case#2019-06-1018:23mssare there any working full-stack examples of using a parallel-reader with fulcro?#2019-06-1018:24msskind of slogging through wiring everything together practically. didn’t realize each resolver in a parallel-reader was going to return a channel, as opposed to the entire parse call returning a single channel to take the query/mutation result from#2019-06-1018:25mssif there’s a way to do that, I’d love to hear it#2019-06-1018:28wilkerlucio@mss in the end you only need to read in the channel that the final parser returns, are you using pathom on the client or in the server?#2019-06-1018:31mssusing it on the server. weird, my parser doesn’t seem to be returning a channel. e.g.
(pathom-connect/defresolver test-resolver [env _]
  {::pathom-connect/output [:something/name :something/type]}
  (let [{:keys []} env]
    {:something/name "hey"
     :something/type :something.type/test-type}))

(defn build-parser [config env-entities]
  (pathom/parser
    {::pathom/env (merge {::pathom/reader               [pathom/map-reader
                                                         pathom-connect/parallel-reader
                                                         pathom-connect/open-ident-reader
                                                         pathom/env-placeholder-reader]
                          ::pathom/placeholder-prefixes #{">"}}
                         env-entities)
     ::pathom/mutate pathom-connect/mutate-async
     ::pathom/plugins [(pathom-connect/connect-plugin {::pathom-connect/register [test-resolver]})
                       pathom/error-handler-plugin
                       pathom/request-cache-plugin
                       pathom/trace-plugin]}))

(def my-p (build-parser {} {}))

(clojure.pprint/pprint (my-p {} [:something/name :something/type]))
#2019-06-1018:31mssthe above returns:
#:something{:name
            #:com.wsscode.pathom.parser{:provides
                                        #{:something/type
                                          :something/name},
                                        :response-stream
                                        #object[clojure.core.async.impl.channels.ManyToManyChannel 0x721582a2 "
#2019-06-1018:31mssI’m definitely missing or misusing something in my configuration, then. just not clear what. on pathom 2.2.14#2019-06-1018:32wilkerlucio@mss the problem is you are using incompatible things#2019-06-1018:32wilkerlucioyou need to use the p/parallel-parser to use the parallel reader#2019-06-1018:32wilkerluciothey have different processing mechanisms#2019-06-1018:33wilkerluciojust replace that, the parser will return a channel, and reading this channel gets you the final response#2019-06-1018:34mssahhh I understand. wonderful#2019-06-1018:34mssknew I had something configured incorrectly. thanks for putting this library out. it’s really a joy to use#2019-06-1018:35wilkerlucioglad to hear you are enjoying the time with it šŸ™‚#2019-06-1311:12Adam Tothbtw, pathom has been great for us too, it's spreading through our projects (only 2 so far)#2019-06-1312:39souenzzopathom is awesome in many ways.#2019-06-1403:24wilkerlucio[com.wsscode.pathom "2.2.15" is out! Changes: * Fix pc/data->shape breaking when data has complex keys * Fix alias on p/join-seq-parallel optimized runs (https://github.com/wilkerlucio/pathom/issues/89)#2019-06-1403:40wilkerlucioalso updated docs for placeholders: https://wilkerlucio.github.io/pathom/#_placeholders thanks to @tony.kay for pointing the missing full examples#2019-06-1413:50uwoAny obvious places to look if the index-explorer isn't working? When I try to connect I get an error like this java.lang.RuntimeException: java.lang.Exception: Not supported: class <one of my resolvers>#2019-06-1414:00wilkerlucio@uwo the issue that happens here is because its likely trying to encode things in transit that it doesn't know about, my suggestion is to use transit default handlers so it encode it anyway (even if it can't properly read on the other side, doesn't matter, those keys are not going to be read anyway)#2019-06-1414:01wilkerlucioI really need to add this to the docs there#2019-06-1414:03wilkerluciospecially because there is a bug in transit where the default write handlers don't work, so there is some code needed to make that work#2019-06-1414:03wilkerlucio
(deftype DefaultHandler []
  WriteHandler
  (tag [this v] "unknown")
  (rep [this v] (pr-str v)))

(defn writer
  "Creates a writer over the provided destination `out` using
   the specified format, one of: :msgpack, :json or :json-verbose.
   An optional opts map may be passed. Supported options are:
   :handlers - a map of types to WriteHandler instances, they are merged
   with the default-handlers and then with the default handlers
   provided by transit-java.
   :transform - a function of one argument that will transform values before
   they are written."
  ([out type] (writer out type {}))
  ([^OutputStream out type {:keys [handlers transform default-handler]}]
   (if (#{:json :json-verbose :msgpack} type)
     (let [handler-map (merge transit/default-write-handlers handlers)]
       (transit/->Writer
         (TransitFactory/writer (#'transit/transit-format type) out handler-map default-handler
           (when transform
             (reify Function
               (apply [_ x]
                 (transform x)))))))
     (throw (ex-info "Type must be :json, :json-verbose or :msgpack" {:type type})))))
#2019-06-1414:03wilkerluciothen you can write with#2019-06-1414:03wilkerlucio
(defn write-transit [x]
  (let [baos (ByteArrayOutputStream.)
        w    (writer baos :json {:handlers transit-write-handlers
                                 :default-handler (DefaultHandler.)})
        _    (transit/write w x)
        ret  (.toString baos)]
    (.reset baos)
    ret))
#2019-06-1414:11wilkerluciook people, new section in the docs, how to fix transit encoding issues: https://wilkerlucio.github.io/pathom/#_fixing_transit_encoding_issues#2019-06-1416:41uwoThank you!!#2019-06-1420:46uwoHas anyone here attempted to integrate vim omni-completion, ctrl-x-o, with pathom-index enabled autocompletion?#2019-06-1420:47wilkerlucio@uwo not that I know, but if you have a running repl with pathom and the index, and you can communicate with that, the impl can be quite easy#2019-06-1420:48wilkerluciothis fn does all the heavy lifting: https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/connect.cljc#L1522#2019-06-1420:49wilkerlucioctx is a vector with a path, examples: [:customer/id], [:my-app/all-users :user/groups]#2019-06-1420:50wilkerluciocache is optional, but can do great on improving performance, just send an atom with a map (and hold this atom, keep sending the same one)#2019-06-1616:42souenzzoI'm trying to resolve [{[:user/id 0] [:user/id #:user{:friends 4}]}] but i'm getting "class java.lang.IllegalArgumentException: Don't know how to create ISeq from: java.lang.Long". Recursive queries isn't supported?#2019-06-1618:22Chris O’DonnellThat doesn't look like a proper query to me. What's the shape of data you're looking to get back?#2019-06-1618:24Chris O’DonnellFYI: You're getting that exception because it expects a subquery (which would need to be a sequence) and is getting 4.#2019-06-1618:40souenzzoIt's a valid query for #fulcro and #datomic It means that I wanna recur 4 times over the "current" query on pathom<>graphql example, there is a Recursive example that shows that notation#2019-06-1618:42souenzzo#2019-06-1618:45Chris O’DonnellMy mistake, then. Thanks for enlightening me!#2019-06-1702:34Chris O’DonnellI'm struggling to get graphql queries with idents at their roots to resolve properly; would love some help/clarification if someone has a better understanding than I do. My query looks like [{[:gql.gift-list/id "id"] [:gql.gift-list/id :gql.gift-list/name]}]. It looks to me like the ident reader adds {:gql.gift-list/id "id"} to the current entity and tries to resolve :gql.gift-list/id and :gql.gift-list/name. However, gql.gift-list/name is not in my oir index, so it seems like the connect reader cannot find a way to resolve it. I've added "gift-list-by-pk", which is the graphql root query I'd like to use for gift list ident queries, to ::pcg/ident-map as {"gift-list-by-pk" {"id" :gql.gift-list/id}}. That doesn't seem to have affected the oir index. From what I've read of the source and the developer guide, it seems like there needs to be some change to the oir index to allow this query to be resolved. Anyone have suggestions/guidance/insight?#2019-06-1703:53wilkerlucio@souenzzo recursiver queries are supported, are you still figuring this one out?#2019-06-1710:05souenzzono. i will try again later#2019-06-1703:54wilkerlucio@codonnell you assumption seems correct, I just found a bit weird the names with dashes on graphql, if you are using the ns suggested by the docs (graphql2) there is no auto transform, so that seems an invalid name for GraphQL#2019-06-1709:16Chris O’Donnell@wilkerlucio I have in my env that I pass to pcg/load-index:
::pcg/mung #(str/replace % \_ \-)
   ::pcg/demung #(str/replace % \- \_)
#2019-06-1713:18wilkerlucio@souenzzo considering the name you used :friends, is that a to-many? if it is I think it's not supported, recursive queries may need to be to-one relationships
#2019-06-1713:21souenzzoIs it a design choice or an implementation detail?#2019-06-1713:38wilkerlucioto be honest I don't remember, it may be possible, what I see is that this have the potential of exploding in number of items very quickly (multiplying every to-many step)#2019-06-1721:19Chris O’Donnell@wilkerlucio if there's anything I can do to help get ident join graphql queries like I mentioned above working, please let me know.#2019-06-1721:20Chris O’DonnellI think it's the last piece I need to get fulcro and pathom interacting well with my graphql server.#2019-06-1721:21wilkerlucio@codonnell would be interesting to debug that, did you tried checking the index after importing the graphql? I wonder if its a bug or just something in the setup, I have uses of that here and they are working, so I wonder if you hit some different space#2019-06-1721:23Chris O’DonnellI did. Will describe them when I get onto a computer this evening.#2019-06-1721:24Chris O’DonnellTldr is the oir index only had input values for my root queries, not for any attributes.#2019-06-1801:19Chris O’Donnell@wilkerlucio Home now; this is what my oir and indents indexes look like. (I abbreviated "little-gift-list.type" as "gql" in previous snippets for brevity.)#2019-06-1801:21Chris O’Donnelllittle-gift-list.type/gift-list is not an ident root query; it does filtering/sorting/pagination on gift list entities. little-gift-list.type/gift-list-by-pk is an ident root query; it takes id as its only param and selects that gift list. little-gift-list.type/gift-list-aggregate includes some aggregation capabilities: max, average, etc. on query results.#2019-06-1801:24wilkerlucio@codonnell my suspect is that the ident map is not mapping with the graphql for some reason#2019-06-1801:24wilkerlucioit has to find it there to work (so it can pull details in)#2019-06-1801:24wilkerluciocan you try doing without any munging and see how it goes?#2019-06-1801:24Chris O’DonnellSure, I'll give that a try.#2019-06-1801:32Chris O’DonnellAlright, I removed the mung/demung config, and it works!#2019-06-1801:33Chris O’DonnellAnd the oir index is properly populated with values#2019-06-1801:43Chris O’DonnellI'd prefer to keep the munging; would you be interested in a PR that fixes the indexes with munging config in place?#2019-06-1807:53maxtI'm trying to use the placeholders, but if I have two, it seems like only one is getting called. It that expected somehow, or am I misunderstanding something? My query looks like this:
[{:company
  [:company/uuid
   {:>/customers
    [:company/uuid
     :company/customers]}
   {:>/employees
    [:company/uuid
     :company/employees]}]}]
#2019-06-1808:01maxtHm, when I make really simple resolvers it does seem to work, so it must be something else.#2019-06-1808:19maxtI found the problem. I had one resolver that returns different results depending on (::p/parent-query env) but pathom don't want to call that resolver twice. Creating separate resolvers seems to solve the problem I had.#2019-06-1812:42wilkerlucio@codonnell sure šŸ™‚#2019-06-1812:43wilkerlucio@maxt resolvers are cached by input+params, so if its only an env change it will be cached, having separated resolvers is a good option, another you can do is set ::pc/cache? false on the resolver to remove caching, I would recommend keep caching unless its a very cheap operation#2019-06-1812:50souenzzo::pc/cache? is a global or resolver option?#2019-06-1812:59kszaboresolver#2019-06-1813:54maxt@wilkerlucio Ok, thanks for clearing that up. I think some of my confusion might come from that I'm using datomic, and it fields weird to disassemble the pull query and define separate resolvers when I can just send the pull-expression directly to datomic. (with validation ofcourse)#2019-06-1813:56maxtI haven't really seen any examples of using datomic and pathom togehter.#2019-06-1813:59wilkerlucio@maxt yeah, with datomic it feels tempting to just delegate everything out (having one resolver to rule then all), but I think its better if you separated those in your mind, what pathom is providing is an access to some keys, it just happens that a lot of time those will just match, and a pull query can get it all. so what I see people doing is create one resolver for each "entity" (in a very loose entity sense), exporting all the scalar values, and one separated for each relationship, since in datomic its quite cheap to pull all these attributes its fine. one for each relationship to reduce traversing. I guess this is a good start point, and later you can tune it up if you see performance opportunities. makes sense?#2019-06-1814:04maxt@wilkerlucio Yes, I think it does make sense. That's kind of where it's leaning already. Thank you for confirming that it's tempting to delgate everything. I guess one way to think of it is that pathom is the query validation (aka authoirzation) layer.#2019-06-1814:14eoliphant+1 Pathom + Datomic is a match made in heaven, but it’s not 1 to 1. but we’ve ended up with a web of resolvers, some datomic ā€˜helper’ funcs that take advantage of the similarities and paper over the differences (handling ā€˜enums’, etc). It’s nice because you can end up following clean arch, etc principles, but with far less work in terms of translation.#2019-06-2002:21Chris O’Donnell@wilkerlucio PR fixing index generation when using mung and demung is up at https://github.com/wilkerlucio/pathom/pull/94. Please let me know if you disagree with the approach or would like to see any changes.#2019-06-2002:42Chris O’DonnellAnd of course, no rush. git deps are a wonderful thing. šŸ™‚#2019-06-2222:37Chris O’Donnell@wilkerlucio Sorry to bother you! A couple of questions when you get a chance: 1. Would it be possible to pass through tempid? at https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/connect/graphql2.cljc#L335? (Or is there a better way to use tempids with pathom connect graphql that I'm missing?) 2. Is there a reason you're not calling str on clojurescript uuids at https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/graphql.cljc#L33? Feels like uuids should be formatted to strings as they are in clojure, but I definitely could be missing something. The formatting is a bit odd right now:
cljs.user> (js/JSON.stringify (clj->js (random-uuid)))
"{\"uuid\":\"f03af9f4-4dec-4c77-b66c-9e1e2ca0f8a4\",\"__hash\":null,\"cljs$lang$protocol_mask$partition0$\":2153775104,\"cljs$lang$protocol_mask$partition1$\":2048}"
cljs.user> (js/JSON.stringify (str (clj->js (random-uuid))))
"\"1e2e14f3-aa82-4579-badb-a3684158d80d\""
Really appreciate all the work you've put into pathom!
#2019-06-2511:25wilkerlucio@codonnell hello, no worries, about 1, I guess that would be ok, but doesn't munging only goes on keys?#2019-06-2511:26wilkerlucio2. I don't remember why is that, maybe just missed, can't think of a case to need to keep as uuids#2019-06-2511:36Chris O’DonnellPerhaps I miscommunicated 1. The reason I'd like to see tempid? passed through there in addition to demung is so tempid? isn't defaulted to (constantly false) when connect builds graphql queries. I couldn't find a way to pass that predicate in, but that could be my unfamiliarity. Wasn't sure what that had to do munging only going on keys, so trying to clarify.#2019-06-2512:05wilkerluciooh, I see, I think its makes sense to forward it down#2019-06-2512:05wilkerluciootherwise, as you found, you can't set it#2019-06-2512:06wilkerlucio@codonnell if you wanna send those changes I would be glad to merge it in#2019-06-2512:31Chris O’DonnellWill do, thanks for your time.#2019-06-2515:37souenzzoWhy ::pc/batch? dont always receive/return "many" ?#2019-06-2516:35wilkerlucio@souenzzo thats for interface/api compatibility, but if you want that pathom has some helpers for you šŸ™‚ you can use the ::pc/transform in combination with pc/transform-batch-resolver to receive consistent input, example:#2019-06-2516:37wilkerlucio
(pc/defresolver batch-sample [env input]
  {::pc/input     #{:some/id}
   ::pc/output    [:some/output]
   ::pc/transform pc/transform-batch-resolver}
  ; input will always be a sequence now, also note ::pc/batch? true will be automatically
  ; add by the pc/transform-batch-resolver helper
  ...)
#2019-06-2521:22mssis metadata on data from a resolver preserved as a parse result is built up? running into this weird case using the parallel parser where some data returned from a root resolver has metadata attached to it. when I actually pull the parse result off the chan returned by the parse call, it doesn’t contain that metadata. kind of at a loss here, and maybe I’m missing something about how core.async preserves/doesn’t preserve metadata under the hood#2019-06-2521:41wilkerlucio@mss, currently there is no assumption on the preservation of the data, there are multiple layers of caching and merging happens in multiple times (specially in the parallel parser), so there is a good chance some meta data is been lost in the process, can you tell me more about what is the use case you have for the metadata? maybe we can find a way around it#2019-06-2521:43mssI’m using ring cookie middleware and depending on the mutation/resolver want to drop a cookie by including a :cookie key in a resolver/mutation result. pathom doesn’t preserve that out of band data if I just include it afaict#2019-06-2521:44mssSo similar to fulcro’s impmementation, I was attempting to add metadata with a response transformation fn onto the result with the data to add to the ring response, parse that result later and feed the raw ring response into the transformation fn#2019-06-2521:45mssIf you have a better idea for how to accomplish including out of band data with a parse result from inside of a resolver I’m all ears#2019-06-2521:54msshttps://github.com/fulcrologic/fulcro/blob/develop/src/main/fulcro/server.clj#L251#2019-06-2521:54msshttps://github.com/fulcrologic/fulcro/blob/develop/src/main/fulcro/server.clj#L332#2019-06-2521:54msshttps://github.com/fulcrologic/fulcro/blob/develop/src/main/fulcro/server.clj#L349#2019-06-2521:55mssHere’s the fulcro implementation as a reference FWIW. What I’m doing looks pretty similar#2019-06-2522:14mssMy inclination is to just put an atom in the env and swap in data in my resolvers that needs to be merged into the response. Deref it post parse and build the complete response#2019-06-2522:53Chris O’Donnell@wilkerlucio PR with uuid and tempid changes at https://github.com/wilkerlucio/pathom/pull/96. (Is pinging you here about PRs actually helpful or just extra noise?)#2019-06-2522:55wilkerlucio@mss doing the atom approach is totally fine, I do that for a lot of things, including the built-in error system šŸ™‚#2019-06-2522:55wilkerlucioyou can use the fn env version to always inject a new one, or write a plugin#2019-06-2522:56mssAppreciate the feedback. Really a lovely library to use, thanks for all the work you put in#2019-06-2522:58wilkerluciothinking a bit about it, in your case, the caller to the parser can send the atom as part of the env, so you have external control over it#2019-06-2715:48wilkerlucio#2019-06-2715:50wilkerlucioThis image demonstrate how much bundle size you can save now by disabling spec for EQL and Pathom, this off course if you are not using spec at runtime. Its worth noticing a lot of the gain comes from avoiding pulling spec things from CLJS (and not so much from the specs themselves), so if you are using spec for something else already, you probably won't save that much by removing specs just from Pathom and EQL.#2019-06-2715:51wilkerlucio@thheller finally done ā˜ļø šŸ™‚#2019-06-2715:56souenzzoWhy closure define was chosen instead of "splited namespaces" like this: https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj#2019-06-2715:59thhellersweet!#2019-06-2716:01wilkerlucio@souenzzo mostly compatibility, I can't remove from the current namespaces without breaking things forward#2019-06-2917:01wilkerluciohello, just did some tweeks to the fonts/styles on the book, also using icons in the captions. I hope you enjoy the new styles šŸ™‚ https://wilkerlucio.github.io/pathom/#2019-06-3022:19eoliphantIf you’re using Pathom with Datomic, a couple new features just dropped, particularly Return Maps that you may find useful. http://blog.datomic.com/2019/06/return-maps-for-datomic-cloud-clients.html#2019-07-0115:54souenzzoI'm getting some errors like this {:clojure.spec.alpha/problems ({:path [:app :com.wsscode.pathom.connect/plan], :pred clojure.core/vector?, :val :my-app/some-key-from-query... Is it a know issue?#2019-07-0116:25wilkerlucionot known, but I guess it may happen as part of a change I did recently, are you on the latest?#2019-07-0116:25wilkerluciobut could also be a legit detection, can you tell me more about this case?#2019-07-0117:08souenzzoOkay. I will try to reproduce a small case]#2019-07-0117:15souenzzo
(let [register [(pc/resolver `foo
                             {::pc/output [::foo]}
                             (fn [env _]
                               (s/explain (s/keys) env)
                               {::foo 33}))]
      plugins [(pc/connect-plugin {::pc/register register})]
      parser (p/parallel-parser {::p/plugins plugins})]
  (async/<!! (parser {::p/reader [p/map-reader pc/parallel-reader]}
                     [::foo])))

#2019-07-0117:17souenzzoIn "real world" it occours when my-function is instrumented
(fn [env _]
  {::foo (http/my-function env ..)}
#2019-07-0118:24wilkerluciothat seems a bug in the specs, let me debug that here, should be simple to fix#2019-07-0118:29wilkerlucioyeah, I think I get it, the problem I'm using ::pc/plan improperly... the ::pc/plan is supposed to be a collection of paths to follow, in this case I'm using ::pc/plan as a single path (instead of a collection)#2019-07-0118:29wilkerlucioI'll see if its easy to replace the usage to ::pc/plan-path, which is the correct name for it#2019-07-0118:30wilkerlucioI started to run things instrumented only recently, thanks for pointing the bad spec#2019-07-0118:38wilkerlucio@U2J4FRT2T fixed: https://github.com/wilkerlucio/pathom/pull/99/files#2019-07-0118:52wilkerlucioplease let me know if you find any other issues with [2.2.17]#2019-07-0119:09souenzzoalready running 2.2.17 in homolog env šŸ˜… also removed some random (dissoc ::pc/plan) from code#2019-07-0116:59kszabohey, I just opened a ticket for a potential improvement for Pathom: https://github.com/wilkerlucio/pathom/issues/98 any1 interested?#2019-07-0117:53dnshttps://medium.com/@den.isidoro/writing-multi-module-monolithic-apps-with-graph-apis-1c095cdaccdf#2019-07-0118:39wilkerlucio[com.wsscode/pathom "2.2.17"] is out! Library Changes: - Looser spec on ::pc/transform - Use contains? instead of find in map reader to support wider range of custom maps - BREAKING: Fixed specs, when processing a plan path, use ::pc/plan-path instead of ::pc/plan in the env, which is the correct name#2019-07-0118:40wilkerluciothat breaking, you are only affected if you happen to use ::pc/plan from the env for any custom extension#2019-07-0315:34souenzzocan I set a timeout to avoid slow resolvers?#2019-07-0315:36kszabothere is ::pc/key-process-timeout#2019-07-0315:38souenzzo::pt/key-process-timeout#2019-07-0315:40kszabodepends on the ns alias :^)#2019-07-0315:42souenzzohttps://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/core.cljc#L1100#2019-07-0315:42souenzzonot sure#2019-07-0315:55souenzzoOnce again it's some odd issue with core.async + pathom + datomic + shadow-cljs solved with -Dclojure.core.async.pool-size=500#2019-07-0316:47thheller@souenzzo do you do any blocking work in core.async loops? shouldn't be required to raise the defaults otherwise#2019-07-0317:10souenzzoAll threads are #pathom + #datomic . No #shadow-cljs ā¤ļø#2019-07-0316:56souenzzoI dont directly use core.async anythere. I'm digging a thread dump, looks like that #datomic is locking all my core.async threads#2019-07-0317:10souenzzoI cant block inside resolvers in parallel-parser, once everything is resolved inside a go block https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/connect.cljc#L1138 I will see if is possible/make sense do this work using go just if the resolver returns a channel#2019-07-0410:21michalhey there, just a general question regarding pathom. how much is that library tied to fulcro? as much as I love the concept behind pathom, I see no real advantage of fulcro over reagent+re-frame, and would rather stick to the later. the thing is that even documentation says We expect that most of our user base is made up of Om Next or Fulcro users, which makes me worry that I will be forced at some point to leak in some fulcro specific ideas into my codebase.#2019-07-0410:54aisamuI think that refers mostly to the query "system" (now called EQL) and related concepts (idents, joins etc.)#2019-07-0411:20kszaboOn your preference to re-frame+reagent I state a new rule based on Greenspun’s tenth rule: Any sufficiently complicated re-frame program contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of Fulcro. šŸ˜‚#2019-07-0410:24kszaboit is not tied at all#2019-07-0410:24kszabothere are some ns-es which require fulcro though for some convenience functions#2019-07-0410:25kszabowhile it’s a great fit for UI’s, we also use it for IPC#2019-07-0410:30michalawesome! if so, I need definitely to give it a try.#2019-07-0411:20kszaboOn your preference to re-frame+reagent I state a new rule based on Greenspun’s tenth rule: Any sufficiently complicated re-frame program contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of Fulcro. šŸ˜‚#2019-07-0411:21kszaboand we are developing/maintaining a re-frame based Google Chrome extension#2019-07-0411:21kszaboalongside a Fulcro SPA#2019-07-0412:57wilkerlucio@souenzzo about the timeout, I suggest not using ::pc/key-process-timeout, this is a last resource parser stop, just for real unexpected scenarios, to do timeout you better use core.async, if you use the async parser or the parallel parser, then you can use the core.async timeout for it, example:#2019-07-0413:00wilkerlucio
(pc/defresolver res-with-timeout [_ _]
  {::pc/output [:foo]}
  (async/go
    (let [timeout-ch (async/timeout 3000)
          [ch res] (async/alts! [(async/go
                                    (do-my-operation-here))
                                  timeout-ch] :priority true)]
      (if (= ch timeout-ch)
        (throw (ex-info "Resolver timeout" {:timeout 3000}))
        res))))
#2019-07-0413:03wilkerluciobut this is less cool way to do it, better options are implementing as a resolver-transform (https://wilkerlucio.github.io/pathom/#connect-transform) or as plugin (https://wilkerlucio.github.io/pathom/#_plugins)#2019-07-0413:05wilkerlucio@michal as @thenonameguy said, not tied at all, it just provide extra helpers for fulcro users, but we are in the process of trying to get most of those out, with Fulcro 3 coming its been complicated to keep these references, so we want for the user to don't pull anything from fulcro unless you are actually using, so fulcro will be a provided dep probably#2019-07-0413:11michal@wilkerlucio having dependency on fulcro as optional would be a great step forward šŸ™‚ btw. I really enjoyed watching your presentations about pathom. very inspiring ones imho.#2019-07-0413:14wilkerluciothanks šŸ™‚#2019-07-0416:36souenzzois it still true? can't find trace in fulcro inspect https://github.com/wilkerlucio/pathom-viz#fulcro-inspect-integration#2019-07-0417:19wilkerlucioit is, but you have to request the trace as part of the query#2019-07-0417:20wilkerlucio@souenzzo add the property :com.wsscode.pathom/trace to your query#2019-07-0417:21wilkerluciothe way I do it most of the time is wrapping the fulcro network, pathom has a helper for that: https://cljdoc.org/d/com.wsscode/pathom/2.2.17/api/com.wsscode.pathom.fulcro.network#trace-remote#2019-07-0417:22wilkerluciobut notice this doesn't work for Fulcro 3, in case you are already using it (the helper thing)#2019-07-1301:34tony.kayF3 allows for a global eql transform that could handle it, bit that would affect all remotes…wrapping the new remote would also be relatively simple#2019-07-0418:19souenzzopfn/trace-remote works with fcn/fulcro-http-remote ?#2019-07-0418:19wilkerlucioworks with any remote from Fulcro 2#2019-07-0418:22souenzzoI'm getting
network.cljs:118 Uncaught TypeError: Cannot read property 'cljs$core$IFn$_invoke$arity$1' of null
    at network.cljs:118
    at fulcro$client$network$progress_routine_STAR__$_progress_fn (network.cljc:287)
    at goog.net.XhrIo.<anonymous> (network.cljc:330)
    at goog.net.XhrIo.goog.events.EventTarget.fireListeners (eventtarget.js:285)
    at Function.goog.events.EventTarget.dispatchEventInternal_ (eventtarget.js:383)
    at goog.net.XhrIo.goog.events.EventTarget.dispatchEvent (eventtarget.js:196)
    at goog.net.XhrIo.onProgressHandler_ (xhrio.js:904)
#2019-07-0418:27souenzzoFull ex
Uncaught TypeError: Cannot read property 'cljs$core$IFn$_invoke$arity$1' of null
    at network.cljs:118
    at fulcro$client$network$progress_routine_STAR__$_progress_fn (network.cljc:287)
    at network.cljc:275
    at goog.net.XhrIo.<anonymous> (network.cljc:324)
    at goog.net.XhrIo.goog.events.EventTarget.fireListeners (eventtarget.js:285)
    at Function.goog.events.EventTarget.dispatchEventInternal_ (eventtarget.js:383)
    at goog.net.XhrIo.goog.events.EventTarget.dispatchEvent (eventtarget.js:196)
    at goog.net.XhrIo.onReadyStateChangeHelper_ (xhrio.js:873)
    at goog.net.XhrIo.onReadyStateChangeEntryPoint_ (xhrio.js:818)
    at goog.net.XhrIo.onReadyStateChange_ (xhrio.js:802)
(anonymous) @ network.cljs:118
fulcro$client$network$progress_routine_STAR__$_progress_fn @ network.cljc:287
(anonymous) @ network.cljc:275
(anonymous) @ network.cljc:324
goog.events.EventTarget.fireListeners @ eventtarget.js:285
goog.events.EventTarget.dispatchEventInternal_ @ eventtarget.js:383
goog.events.EventTarget.dispatchEvent @ eventtarget.js:196
goog.net.XhrIo.onReadyStateChangeHelper_ @ xhrio.js:873
goog.net.XhrIo.onReadyStateChangeEntryPoint_ @ xhrio.js:818
goog.net.XhrIo.onReadyStateChange_ @ xhrio.js:802
XMLHttpRequest.send (async)
goog.net.XhrIo.send @ xhrio.js:632
fulcro$client$network$xhrio_send @ network.cljc:30
(anonymous) @ network.cljc:334
fulcro$client$network$transmit @ network.cljc:103
(anonymous) @ network.cljs:116
fulcro$client$network$transmit @ network.cljc:103
(anonymous) @ client.cljs:187
fulcro$client$network$transmit @ network.cljc:103
fulcro$inspect$client$handle_devtool_message @ client.cljs:320
(anonymous) @ client.cljs:46
postMessage (async)
(anonymous) @ content-script.js:96
#2019-07-0418:31souenzzooccurs just when there is no p/trace-plugin on server šŸ™‚#2019-07-0418:23souenzzojust when I'm using 'send to query'#2019-07-0915:01joseis it possible to render pathom-viz standalone? instead of inside a card#2019-07-1020:37souenzzoI sent the wrong channel#2019-07-1020:46geraldodevAny tips , for using pathom with clojure client (not cljs) ?#2019-07-1023:12wilkerlucio@jlle yes, you can mount the component as a Fulcro app, you just have to setup the remote properly, are you familiar setting up fulcro apps?#2019-07-1023:12wilkerlucio@geraldodev given the parser is a fn, using in clojure is simple as calling it šŸ™‚ is there any specific issue you are hitting?#2019-07-1023:32geraldodev@wilkerlucio no specific issue. I was looking for some examples, Found namespace diplomat on pathom that has clj-http client.#2019-07-1023:42wilkerlucio@geraldodev that diplomat is more about giving a compatibility layer to call http across clj/cljs, given different implementations will be needed on each, that's an interface to try to wrap the http needs#2019-07-1100:25geraldodevI see that you've defined specs for http keys and build-request-map uses that spec'ed namespaced keys to build the request that are enforced at time of request and request-async with s/assert#2019-07-1116:08wilkerlucio@geraldodev yeah, this is just for the common http call interface (from inside resolvers), this namespace has nothing to do with using the parser, makes sense?#2019-07-1120:31geraldodev@wilkerlucio totally#2019-07-1307:04jose@wilkerlucio I'm not that familiar with fulcro, but I think I'll figure it out looking into the Fulcro template. But is not clear to me what is the main entry point for pathom-viz, that needs to be mounted as a Fulcro app#2019-07-1317:25wilkerlucioevery component is a potential entry point, that's part of the fulcro design, but usually the composed things are at the end of the file, usually I keep one main component per file in pathom-viz#2019-07-1317:25wilkerlucioI'll try to get that more clear in the docs, thanks for the feedback#2019-07-1323:06wilkerlucio@U4NGX0FHN oh, I just reminded I have docs for that in the pathom book: https://wilkerlucio.github.io/pathom/#_stand_alone_app#2019-07-1410:52josethanks for pointing me to the docs, I missed it. I thought that QueryEditor was the main component, good to know that is IndexExplorer I'm still getting some errors, but I hope to figure it out#2019-07-1410:52josethanks for pointing me to the docs, I missed it. I thought that QueryEditor was the main component, good to know that is IndexExplorer I'm still getting some errors, but I hope to figure it out#2019-07-1605:21tony.kayWilker, https://wilkerlucio.github.io/pathom/#_parsing_environment_and_the_reader ā€œThe parsing environment is simply a map that carries along data while parsing (and can be augmented as you go).ā€ care to elaborate on ā€œand can be augmented as you goā€ā€¦as far as I know the only way to do that is by stashing an atom in there…is there some special return value for resolvers that will put extra stuff into env for the rest of the parse?#2019-07-1616:38wilkerlucio@tony.kay yes, there is a special thing you can use to augment env, sorry missing docs, I'll add in a bit and let you know#2019-07-1616:38wilkerluciojust a heads up, you can only do that when changing context (you can't change env for a sibling, and that would be a mess given pathom doesn't garantee the order in which attributes will process)#2019-07-1616:39wilkerlucioone way to change data for siblings is having an atom in the env, then you can change anyway (but risking concurrency issues)#2019-07-1814:24uwoHow do we issue queries from Fulcro that satisfy resolvers with multiple required inputs (i.e. how to leverage :pathom/context)? I know that using load with a component that has an ident will issue an ident query, but what's the approach for adding more info to that? (Also, I'd like to add it to :pathom/context, not to env params, which I have figured out šŸ™‚ )#2019-07-1816:54lepistaneHello! i recently saw https://www.youtube.com/watch?v=UvJEBMOtayk and i got really interested in this. I don't fully get the concept and i want to get my hands dirty Do you have any project in mind that would allow me to use this concept and explore it little bit more? Are there any examples of simpler projects? sorry if the question sounds stupid but i am not sure how to ask it right i feel like i am in the 'i dont know what i don't know zone' but this concept is really really interesting to me#2019-07-1819:28souenzzoI have some sample/demo apps that I usually develop with pathom/fulcro for some PoC or try something new https://github.com/souenzzo/graph-demo#2019-07-1906:30lepistaneoh wow there are a lot of stuff here how does your software look like when this is demo šŸ˜„ thanks for sharing#2019-07-1923:20souenzzoPS: Although I am a #datomic user, I think that I'm still a #crux noob.#2019-07-1819:45wilkerlucio@lepistane welcome šŸ‘‹ , I think it can be interesting to write any kind of integration system, something to combine data from multiple sources are a very nice fit, and you can do that just implementing the parser and running queries, and then later, if you like, you can plug Fulcro app to make it visual, makes sense?#2019-07-1907:13lepistane@wilkerlucio thanks! ye it makes sense side question do i have to use fulcro? I have heard of it but i've never used it i think it would be a burden to learn that as well as work with eql/pahtom i am comfortable with re-frame/reagent would that work too ?#2019-07-1911:55myguidingstar@lepistane technically EQL/Pathom is indepedent from client frameworks#2019-07-1911:57myguidingstarbut I would say learning Fulcro is just mind blowing. Once you master the state/query/ident trinity, you'll find re-frame kinda repetitive#2019-07-1912:00myguidingstarit liberates you from distracting details like handling http statuses and callbacks which are ubiquitous in re-frame apps#2019-07-2109:22joseI'm still trying to make the pathom-viz query editor a standalone app. I notice that the simple-parser-demo card expects a local Pathom parser, but I already have a http server listening for Pathom queries, so I'm trying to make the requests to the server. I think I should update this: https://github.com/wilkerlucio/pathom-viz/blob/abb4006be4181ebf4c803a1e07edbe4a8ff790c1/src/core/com/wsscode/pathom/viz/workspaces.cljs#L31-L34 to make http requests, but I'm lost, not sure how. Any suggestions?#2019-07-2112:15wilkerluciohello @jlle, from the snippet you sent I'm remembering that the query runner uses a different remote name, that's this pv.query-editor/remote-key, did you tried settings this remote to be your remote?#2019-07-2114:13jose@wilkerlucio you mean something like
:networking
{pv.query-editor/remote-key
 (p.network/pathom-remote "")}
?
#2019-07-2114:15wilkerlucioclose, but you have to use the regular fulcro network if you want to use the standard HTTP remote, the pathom-remote is a wrapper to turn local async parsers in remotes#2019-07-2114:24wilkerluciooh, sorry, I just realized it will require some more effort to wrap the http call#2019-07-2114:24wilkerluciothere is a problem in the current implementation, you actually have to use an async parser interface to use the network it provides#2019-07-2114:25wilkerluciobut its not so bad, you can wrap the remote with a parser interface, which is: [env tx] => channel#2019-07-2114:26wilkerluciolet me try to get a demo, that will help updating the docs also#2019-07-2114:27wilkerlucioor, do a fetch call from the parser interface#2019-07-2114:35wilkerlucio@jlle something like this:#2019-07-2114:35wilkerlucio
(defn transit-read [x]
  (-> (transit/read (transit/reader :json) x)))

(defn transit-write [x]
  (-> (transit/write (transit/writer :json) x)))

(defn http-request-parser [url]
  (fn [env tx]
    (go-catch
      (let [{::p.http/keys [body]}
            (<? (p.http/request {::p.http/driver       p.http.fetch/request-async
                                 ::p.http/url          url
                                 ::p.http/content-type ::p.http/transit+json
                                 ::p.http/method       ::p.http/post
                                 ::p.http/headers      {}
                                 ::p.http/form-params  (transit-write tx)}))]
        (transit-read body)))))

; networking
:networking
{pv.query-editor/remote-key
 (p.network/pathom-remote
   (pv.query-editor/client-card-parser (http-request-parser "")))}
#2019-07-2115:43jose@wilkerlucio thanks, it looks better, I see that pathom queries are done to the server šŸ™‚ For some reason the results are not displayed on the card. But I can see the proper response on the chrome devtools network tab. I don't have time now, but I'll try to debug it later#2019-07-2213:49josethe problem was that I was still using the pp/profile-plugin instead of the new p/trace-plugin, now all works in the card, but I'm still having problems rendering the editor as a standalone app, I tried to adapt the example in the docs ( https://wilkerlucio.github.io/pathom/#_stand_alone_app ):
(fp/defsc Root
  [this {:keys [ui/root]}]
  {:query [{:ui/root (fp/get-query pv.query-editor/QueryEditor)}]}
  (pv.query-editor/query-editor root))

(def root (fp/factory Root))

(defn ^:export init []
  (let [app (fc/make-fulcro-client
              {:client-did-mount
               (fn [app]
                 (df/load app [::pv.query-editor/id "singleton"] pv.query-editor/QueryEditor
                   {:target [:ui/root]}))
               :networking
               {pv.query-editor/remote-key
                (p.network/pathom-remote
                  (pv.query-editor/client-card-parser (http-request-parser "")))}})]
    (fc/mount app Root (js/document.getElementById "app"))))
but I guess I'm missing something, since I get multiple warnings and error on the js console
#2019-07-2213:52wilkerlucio@jlle any reason for not switching to trace? and errors are you seeing?#2019-07-2214:13joseI switched to trace, that fixed the problems with the card#2019-07-2214:14joseI got this warning: [fulcro.client.primitives] component com.wsscode.pathom.viz.query-editor/QueryEditor's ident ([:com.wsscode.pathom.viz.query-editor/id nil]) has a 'nil' second element. This warning can be safely ignored if that is intended. #2019-07-2214:15joseand this error when I try to execute a query: [fulcro.client.impl.application] Use of invalid remote(s) detected! #{:remote} #2019-07-2214:16josealso, the layout seems broken, not as beautiful as the card, and I get an CodeMirror error: Error setting up CodeMirror TypeError: Cannot read property 'split' of null#2019-07-2214:43wilkerlucio@jlle for the layout I guess you will have to upsert the css, check fulcro docs on how to do it#2019-07-2214:43wilkerlucioand for the load, I see you need a change there, you need to point to the correct remote, add :remote pv.query-editor/remote-key to do map in the load call#2019-07-2214:44wilkerlucioactually, maybe you don't need this load at all#2019-07-2215:51josethanks, I managed to upsert the css, now looks better. But I guess that the component is not properly initialized, apart of the errors I posted, I noticed that the :initial-state is not applied (e.g.: Request trace checkbox should be enabled, but is not )#2019-07-2216:18joseI think this error is related, if I click on 'Refresh index` I get Uncaught Error: Assert failed: (or (util/ident? server-property-or-ident) (keyword? server-property-or-ident))#2019-07-2216:24wilkerluciohumm, do you have fulcro inspect? not sure if you are familiar with fulcro apps, but inspecting the DB can be helpful to find were the data link is broken#2019-07-2216:24wilkerlucioabout the initial load, if you want to load the index at start, instead of doing a manual call you can call this fn: https://github.com/wilkerlucio/pathom-viz/blob/master/src/core/com/wsscode/pathom/viz/query_editor.cljs#L65-L71#2019-07-2220:28jose@wilkerlucio thanks again, fulcro inspect helped me. I think that finally I got the query editor properly working as a stand alone app šŸ™‚ Would you be interested in a PR to pathom-viz to add a build for the standalone app?#2019-07-2220:29wilkerluciosure šŸ™‚#2019-07-2220:29wilkerlucioI'm also planning on adding new docs soon for Pathom Viz, following the format in http://edn-query-language.org#2019-07-2220:29wilkerlucioso we could have a proper tutorial there, if you like to write something in asciidoc format would be great!#2019-07-2220:32josesure, I want to test what I have and do some clean up and I will do a PR#2019-07-2308:16cjmurphyWhen code is running from a defresolver, have asserts (as in a call to the assert function, no spec or ghostwheel anything) been disabled?#2019-07-2314:41wilkerlucio@cjmurphy pathom does nothing special around then, usually that should be captured by the error handler plugin, what are you experiencing?#2019-07-2314:43cjmurphyThey are just being ignored. I should find out where the error handler is. I just copy/pasted the Pathom parser.#2019-07-2314:44cjmurphyp/error-handler-plugin looks like it.#2019-07-2412:59cjmurphyIf you think a resolver should be being called, but it is not, is there some place to look for the reason why it was not called?#2019-07-2413:40cjmurphyThe desired resolver is being called now. I changed how load is done from Fulcro to be into an ident (worked) rather than into a query (did not). The query was getting into the Pathom parser on the server, then just being ignored.#2019-07-2415:00wilkerlucio@cjmurphy maybe the path you are trying to do is not reachable. pathom will not work until it knows it can get to the result, so, if it tries to compute the paths but none is available (not enough input) then nothing will run, that may explain why you can it with an ident join (since you provide the data point) but not out of it. makes sense?#2019-07-2415:00wilkerlucioare you using the tracing feature? if you do, you can click on the trace line, that will log event details to the console, you can see if a plan was properly computed looking over there#2019-07-2415:50cjmurphyThe ident join thing kind of makes sense to me because the ident is [:conflicting-match/id 1], and that is something that the output query needs, and the map that is returned has a kv pair of :conflicting-match/id 1. At the moment all my queries are global, output only - so I don't feel I'm 'into' Pathom yet. Things are working at present, for queries and mutations against Datomic. No I'm not using the tracing feature yet. Maybe using it will help me see the benefits better. To me pull syntax is conveniently generated by the Fulcro client and sometimes (not this particular case) I want to pass it directly through. I've found a way to do that (putting it into env), in which case I'm bypassing Pathom.#2019-07-2418:54wilkerlucioif you don't require inputs then it may be a problem on how the load is triggered, fulcro really relies on idents, its better if you adapt yourself to have idents everywhere, this way your things will scale nicely#2019-07-2423:50cjmurphyI do have idents everywhere in the Fulcro app. I query Datomic through Pathom behind the web server, so the data ends up in Fulcro app client state at particular ident locations. The resolvers are 'global', in that they only have ::pc/output specified (not ::pc/input).#2019-07-2501:02cjmurphyOn a completely different subject now. In the REPL all I see is "before silent failure".#2019-07-2501:06cjmurphyThis is the parser used when getting the silent failure. I've commented out two plugins to try to help, but to no avail.#2019-07-2502:05cjmurphyPerhaps one of the libraries Pathom uses? Ghostwheel?#2019-07-2503:26cjmurphyOr perhaps if I didn't run the async parser. With core.async errors will be thrown on threads and won't be seen on the REPL - that is expected I would have thought.#2019-07-2518:24wilkerlucio@cjmurphy did you figure how to get the errors? Ghostwheel is oly a dev dep, but its not currently used in pathom at all#2019-07-2522:26cjmurphyI tried very briefly to great a non-async parser. When that wasn't successful I decided to create and use my own crude version of a parser. So for now I'm getting on. I'm sure it is in the docs how to get at errors, just too straightforward to go with my own approach as was mostly ducking around Pathom anyway. I'll re-introduce Pathom soon, first up for resolving mutations.#2019-07-2718:37mdhaneyI’m using pathom with Datomic, and I have a couple of questions on how to optimally do things. 1) I started out injecting the current db into the environment, because in general you want each resolver using the same snapshot of the db. This breaks with mutation joins, however, because when the resolvers are run on the returned query, they are using the db from before any transactions that ran in the mutation. To get around this, you could just grab the current db in each resolver, but then you run the risk of data changing between one resolver and another. What I ended up doing is creating a global resolver that injects the current db under a special key. Then in all my ā€œcontextā€ resolvers (i.e. ones that take an ident and establish the context of the graph traversal) I set this special db key as an input in addition to the ident input. When I do a mutation, I set the special db key to the :db-after value from the transaction, so then subsequent resolvers are using the updated database. This works great, but I just wanted to know if there’s any easier way to do this, since I’m not 100% up to speed on all the features in pathom.#2019-07-2718:41eoliphantI did something similar. talked to @wilkerlucio about this and there’s no easy answer. you generally want the nice stable db that datomic gives you but for a single mutation join you need the new db value, never mind once things are going in parallel#2019-07-2718:44mdhaneyYeah, the only other way I can think of to do it would be to update the db in the environment after the mutation is processed but before the join query is parsed. But I didn’t see anything in the docs to indicate it was possible to change the environment like that.#2019-07-2718:48eoliphantyeah I created a couple funcs that create an atom that gets passed in with the db and conn, then a plugin, that would swap in a new db value if a mutation was processed. Was trying to make it smarter to say only do it if there was in fact a mutation query, but never got around to finishing that#2019-07-2718:50mdhaneyHmm, that’s a good idea. I’ll keep that in mind if I run into any problems with my current approach.#2019-07-2718:52eoliphanti’ll scare it up and send it over when i get a chance#2019-07-2719:10mdhaneyThat would be great. I wouldn’t mind learning more about plugins anyway, since all this Datomic stuff could be a useful plugin to package up and reuse on future projects.#2019-07-2718:52eoliphantlearned more about plugins doing it šŸ˜‰#2019-07-2718:54eoliphantit presents a sorta ā€œNP hardā€/Morpheus lol problem ā€œWhat is current?ā€œ. Since the whole point on the one hand is let resolvers go off and do their thing. But on the other, there’s a sorta implicit outside expectation of ā€˜consistent state’ in terms of the response#2019-07-2719:07mdhaney2) second question You have an entity with a bunch of ā€œregularā€ attributes (i.e. things that don’t need any special processing or require a separate resolver) - what’s the best way to structure the resolvers? What I’ve been doing is 1 resolver that does a Datomic pull for all the possible fields and returns them, and my understanding is that pathom will just pick the fields it needs and ignore the rest. Not bad, and on Cloud probably the way to go so you don’t have to many round trips. I’m on Peer, though, and it does seem inefficient to pull say 30-40 fields if the client is only asking for 2. So one thing I could do is have the resolver that resolves the ident return the actual Datomic entity. Then for each field have a resolver that takes the entity as input and returns that field. This could easily be wrapped up in a couple of macros so it’s not a pain creating all those little resolvers. I’m trying to decide if it’s worth it or not, performance wise. It’s cheap to retrieve the entity, and then the attributes are pulled lazily, so pathom can quickly get each separate attribute in parallel, vs having to wait for all the attributes to be pulled before it can get any of them. I might have to just benchmark and compare the two approaches, but if someone has already tried, that would save me the time. šŸ˜‰#2019-07-2719:13eoliphantI’d been playing around with some datomic helper functions that actually only query for the fields requested in the resolver.#2019-07-2719:14eoliphantthey worked fairly well, though i was running into some limits of generalization lol#2019-07-2719:14eoliphantsomething like that might be the ticket for you as well.#2019-07-2719:15eoliphantand of course even with what you’re doing, once the segments or whatever are ā€˜hot’ in the peer, then datomic generally does a good job of keeping them up to date, so perhaps other than the first hit, the perf wouldn’t be too bad? we’ve moved to clould but still have a few on-prem based services#2019-07-2719:17mdhaneyYeah, we started with Cloud on this project, but then we saw the pricing for the production deployment - yikes!#2019-07-2719:19eoliphantyeah, the min is around $200 a month. but if you don’t need it, there are some tricks you can use like a lambda warmer that make solo an option for ā€˜production’ deployments, you just don’t get the scalability reliability stuf#2019-07-2719:22mdhaneyReally? The numbers I saw were around $450/month. That was about half for instances (the minimum they recommended) and the other half was the license. With Peer, we can start around $125/month and don’t have to worry about the license for the first year.#2019-07-2719:24eoliphanthaving said that, with the ions, if you can move your code over your TCO numbers might line up. Hmm, yeah shouldn’t be that high. the principal cost is the i3's which are a little over 100/month/each. and the license was just a bit more i thought. Let me double check our billing when i get a chance#2019-07-2719:26eoliphantyeah I ended up moving our on-prems to i3's with valcache enabled#2019-07-2719:27mdhaneyI was thrown off because the pricing calculator on AWS marketplace was broken. When I saw the real price, my jaw dropped. It was basically the same as the instances.#2019-07-2719:28eoliphantyeah you’re right my bad. on an i3.large it’s 226/month per#2019-07-2719:29mdhaney:white_frowning_face:#2019-07-2719:30eoliphantgotta double check#2019-07-2719:30eoliphanti know there are more options for running query group nodes#2019-07-2719:30mdhaneyI’m sure that setup has a lot of capacity. The thing is, my customer is bootstrapping this app and needs to get to where ad revenue covers costs ASAP. So we need as cheap as possible and then scale up as needed.#2019-07-2719:31eoliphantbut not sure if you can say pick a non i3 for the transactors#2019-07-2719:31eoliphantyeah totally get that#2019-07-2719:31eoliphantand that’s a bit more to recoup, even factoring in the ability to eliminate app servers etc#2019-07-2719:32mdhaneyThere are a lot of things I like about Peer, so not too bad to go back. It definitely made it easier to use web sockets.#2019-07-2719:33eoliphantyeah it’s only http://i3.xxx for production transactors#2019-07-2719:33eoliphantyeah there are a few things i miss for sure with the peer api#2019-07-2719:33eoliphantd/filter, etc for sure#2019-07-2719:34mdhaneyTransaction log is useful too. I’m using it to send push updates to clients through web sockets.#2019-07-2719:36eoliphantyeah, i sorta get why it’s not in cloud, as you can pretty easily roll your own, we did, that shoots stuff off to AWS whatever, but still would have been nice to have native support. Started off doing Websocket support by streaming log stuff out, then through AWS IOT, etc etc. A bit roundabout but it worked. But now they have the HTTP direct stuff, so looking at simplifying that bit#2019-07-2722:54Mark AddlemanI'm curious what you're approach to emulating the Tx Log is. Do you just wrap datomic's transact fn with something that pushes the results to a queue?#2019-07-2801:28eoliphantI have a lambda that you call, it checks the offset, reads the log entries since the offset calling fns we've configured, then saves the offset. You can only get per minute resolution using cloud watch events, but theres a trick you can do with step functions to say call it every 10 secs etc#2019-08-0123:54daniel.spaniel@U380J7PAQ are you doing websocket things with datomic ion server ?#2019-08-0422:22eoliphantHey @UEC6M0NNB, sorry was traveling, yeah I’m actually working on tryna get websockets/SSE working with HTTP Direct as we speak#2019-08-0422:42daniel.spanieloh boy .. if you get something let me know . i would love to see how you did it ( we have been puzzled for a while on this one )#2019-08-0422:53eoliphantWill do šŸ™‚#2019-07-2815:27daniel.spanielwhat is a good way to get the pathom parser to throw exceptions for certain mutations i am doing in fulcro. is there a way i can merge {::fail-fast true} into the pathom request , since I only want to throw the exception on certain mutations or is there a way I can include that in the fulcro ui request ?#2019-07-2815:31daniel.spanieli was trying this#2019-07-2815:31daniel.spaniel
(defmutation create-invoice
  [{:keys [conn]} {invoice :entity}]
  {::pc/sym 'dataico.ui.invoice.actions/create-invoice
   ::p/fail-fast true}
  (create-invoice* conn invoice)
  {})
#2019-07-2913:44wilkerlucio@dansudol hello, had you see this docs before? https://github.com/fulcro-legacy/fulcro-incubator#pessimistic-mutations#2019-07-2913:45wilkerluciothey have some info on how to handle the errors, Pathom will avoid breaking the whole parser as most as it can, so partial failures can be handled, so you have to handle the errors as part of a regular response#2019-07-2913:45wilkerluciothe other error mechanism in fulcro is intended for hard failures (like network failures)#2019-07-2913:48daniel.spanielI have read that a few times @wilkerlucio and am trying to figure out how to get that pathom reader error out of the pm response ( i will keep rereading that ) and I think it makes sense to not throw network error for every little failure in pathom ( that would be tedious ) , but in this case without that error I will never hit the fulcro ( error-action <<< ) condition#2019-07-2913:49daniel.spanielI am fine with handling the error in the (ok-action ) condition of the pm mutation but i just don't know how to get that error ( I put a nice message when I threw the error in the transaction )#2019-07-2913:49wilkerluciook, maybe whats missing is how to proper expose the errors, by default pathom just spits the error string, but you can override that with ::p/process-error#2019-07-2913:50wilkerluciolet me check, I'm not sure if that's properly documented#2019-07-2913:50daniel.spanielok .. thanks#2019-07-2913:55wilkerlucio@dansudol here https://wilkerlucio.github.io/pathom/#_error_handling#2019-07-2913:59wilkerluciobasically fulcro expects an specific key to determine a mutation repsonse was an error, you can use your custom fn in ::p/process-error to provide it#2019-07-2914:04daniel.spanielgotcha Wilker .. I will go with that .. thanks alot for taking time to show#2019-07-3100:37tony.kay@wilkerlucio I was working with @dansudol on getting unions working with connect…it looks like pathom is seeing union queries as recursion limits instead of unions.#2019-07-3100:38tony.kaywhenever you get time, perhaps you could look…we’ll try to narrow it down further#2019-07-3111:49wilkerlucio@tony.kay did you figure it out?#2019-07-3111:55daniel.spanielwe never did Wilker, we were hoping you had an idea#2019-07-3112:36wilkerluciosure, can you send a minimal case of what you are trying so I can reproduce here?#2019-07-3113:09daniel.spanielits basically where you have an entity like company which has company/id and this company entity ( using datomic ) has a bunch of attributes, and lets say one of them is company/thing#2019-07-3113:09daniel.spaniellet's say company/thing holds pointer to different types of entities like pots or pans#2019-07-3113:10daniel.spanielso if i wanted to query for companies and their thing i would say#2019-07-3113:10daniel.spaniel
{:company/id [:company/thing {:thing/pot [:color :name] :thing/pan [:size :name]}]}
#2019-07-3113:11daniel.spanielbecause i want to get maybe a thing/pot ( with its attributes ) or thing/pan with different ones from the company/thing reference#2019-07-3113:12daniel.spanielwith fulcro we found that we could compose this query , but pathom did not want to parse it#2019-07-3113:57wilkerlucio@dansudol thanks, but I mean some full example trying to use with pathom and the results your are seeing#2019-07-3113:58wilkerlucioI ask because I do use the union feature on a daily basis, so I wonder what you guys may be doing different#2019-07-3114:10daniel.spanielI not sure how to show more @wilkerlucio because the query we were doing was just exactly like I put above, and we never even got to the resolver stage, pathom just said "no thanks pops I not even looking at that query" with an exception with the "recursion limits" message#2019-07-3114:11daniel.spanielif what I posted above is not proper union query then let me know, or maybe you have something going on in your parser ? that is different than mine .. maybe our parser setup needs some special sauce ?#2019-07-3114:19daniel.spanieli can screen share if it is easier to see the full picture because i know just sending a query is kinda lame ( but not sure how else to show it)#2019-07-3114:20wilkerlucioI mean na example creating a parser, the resolvers and triggering a query, so I can run it and see the same thing you are seeing#2019-07-3114:20wilkerlucioa minimal reproduction also helps to drill down the potential problem#2019-07-3114:20wilkerluciolet me write one real quick here#2019-07-3114:21daniel.spanielin our case it is easy because the resolvers never were touched .. the parser refused to run the query#2019-07-3114:21wilkerluciothe query you sent me is the exact query you are issuing?#2019-07-3114:22daniel.spaniel
(defn get-parser []
  (let [real-parser (p/parallel-parser
                      {::p/mutate  pc/mutate-async
                       ::p/env     {::p/reader               [p/map-reader
                                                              pc/parallel-reader
                                                              pc/open-ident-reader
                                                              p/env-placeholder-reader]
                                    ::p/placeholder-prefixes #{">"}}
                       ::p/plugins [(pc/connect-plugin {::pc/register (vec (vals @pathom-registry))})
                                    (p/env-wrap-plugin
                                      (fn [{:keys [conn] :as env}]
                                        (cond-> (merge env {:config config})
                                          (not conn) (assoc :conn (get-conn)))))
                                    (preprocess-parser-plugin log-requests)
                                    (p/post-process-parser-plugin p/elide-not-found)
                                    p/request-cache-plugin
                                    p/error-handler-plugin
                                    p/trace-plugin]})
        ;; NOTE: Add -Dtrace to the server JVM to enable Fulcro Inspect query performance traces to the network tab!
        trace?      (not (nil? (System/getProperty "trace")))]
    (fn wrapped-parser [env tx]
      (async/<!! (real-parser env (if trace?
                                    (conj tx :com.wsscode.pathom/trace)
                                    tx))))))

(defstate parser :start (get-parser))
#2019-07-3114:22daniel.spanielthat is our parser setup#2019-07-3114:23daniel.spanieland we actualy tried something exactly like that ( that same form ) and what was nice ( so to speak ) was that we could do any union looking query ( even if the attributes were non existent like#2019-07-3114:24daniel.spaniel
{:company/id [:company/thing {:noodle [:w :blah] :rice [:a :b]}]}
#2019-07-3114:24daniel.spanielif we ran anything that looked like union .. pathom would not parse it#2019-07-3114:24wilkerluciobecause a map is not a valid query#2019-07-3114:24wilkerlucioqueries must be vectors#2019-07-3114:24wilkerluciothis can't be your root#2019-07-3114:25daniel.spanielso how does union work then because fulcro sets up a union query to look like that ( according to Tony ) and our exeriments yesterday#2019-07-3114:25wilkerluciothat can be a join of something, but can't be the root#2019-07-3114:25wilkerlucioeg: [{[:some/join 123] {:branch/a [...] :branch/b [...]}}]#2019-07-3114:26daniel.spanielsure sure .. you mean i should have typed#2019-07-3114:26daniel.spaniel
[{:company/id [:company/thing {:noodle [:w :blah] :rice [:a :b]}]}]
#2019-07-3114:26wilkerluciothat's wrong too#2019-07-3114:27daniel.spanielaaah .. yeah .. i see what you mean#2019-07-3114:27daniel.spanielbut your assuming we know the join id ( which we don't )#2019-07-3114:27wilkerluciodoesn't need to be an ident join#2019-07-3114:27wilkerluciobut must be a join#2019-07-3114:27wilkerlucio[{:named-join {:branch/a [...] :branch/b [...]}}]#2019-07-3114:28daniel.spanielright .. exactly#2019-07-3114:28daniel.spaniel
{:company/id [:company/thing {:noodle [:w :blah] :rice [:a :b]}]}
#2019-07-3114:28daniel.spanielthis is what we were doing minus the []#2019-07-3114:28daniel.spaniel
[{:company/id [:company/thing {:noodle [:w :blah] :rice [:a :b]}]}]
#2019-07-3114:28wilkerlucioI just got this example working here:#2019-07-3114:28daniel.spanielbut assume the vector was there#2019-07-3114:28wilkerlucio
(quick-parser {::pc/register [(pc/resolver 'thing
                                  {::pc/output [{:get-thing [:a/id :b/id]}]}
                                  (fn [_ _]
                                    {:get-thing {:a/id "A"}}))

                                (pc/resolver 'a
                                  {::pc/output [:a/name]}
                                  (fn [_ _]
                                    {:a/name "A name"}))

                                (pc/resolver 'b
                                  {::pc/output [:b/name]}
                                  (fn [_ _]
                                    {:b/name "B name"}))]}
    '[{:get-thing
       {:a/id [:a/name]
        :b/id [:b/name]}}])
#2019-07-3114:29wilkerlucioquick parser is just a helper I use for testing, it generates a basic parser and run it#2019-07-3114:29wilkerlucioin case you need for details:
#2019-07-3114:29wilkerlucio
(defn quick-parser [{::p/keys  [env]
                        ::pc/keys [register]} query]
     (let [parser (p/parallel-parser {::p/env     (merge {::p/reader               [p/map-reader
                                                                                    pc/parallel-reader
                                                                                    pc/open-ident-reader
                                                                                    p/env-placeholder-reader]
                                                          ::p/placeholder-prefixes #{">"}}
                                                         env)
                                      ::p/mutate  pc/mutate-async
                                      ::p/plugins [(pc/connect-plugin {::pc/register register})
                                                   p/error-handler-plugin
                                                   p/request-cache-plugin
                                                   p/trace-plugin]})]
       (async/<!! (parser {} query))))
#2019-07-3114:29daniel.spanielman .. this is neat parser trick#2019-07-3114:29wilkerluciorun examples#2019-07-3114:30daniel.spanielthis is neat o .. i will try and let you know#2019-07-3114:30wilkerlucio(noticed I changed the get-thing resolver across the runs to demonstrate the branch selection)#2019-07-3114:30wilkerluciowhat you find new there?#2019-07-3114:30daniel.spanielyes#2019-07-3114:30daniel.spanielmaybe you declared resolver like
{:get-thing [:a/id :b/id]}
#2019-07-3114:30daniel.spanielwith ::pc/out#2019-07-3114:31daniel.spanielso maybe pathom was able to recognize ?#2019-07-3114:31wilkerlucioyeah, you need that, otherwise pathom can't know what to do (or how to auto-complete)#2019-07-3114:31wilkerluciothere are other ways do it, its valid to also do in a flat way#2019-07-3114:31daniel.spanieli think we were missing that .. 80% sure#2019-07-3114:32daniel.spanielfor that
::pc/output [{:get-thing [:a/id :b/id]}]
#2019-07-3114:33wilkerluciocool, I hope that can make things move for you#2019-07-3114:33wilkerlucioif not, lets keep on it šŸ™‚#2019-07-3114:33daniel.spanielshould i list all the attributes besides id ? like [:a/id :a/name :a/blah :b/id. .etc ]#2019-07-3114:34daniel.spanielfor sure thanks W .. i will run this and check it#2019-07-3114:35wilkerluciolist all the things you intend to return from this resolver#2019-07-3114:35daniel.spanielgot it šŸ™‚#2019-07-3115:33tony.kayThe problem was with the output spec. I assumed it should look like a union, but your example cleared it up: a vector of possible IDs instead of a union query in ::pc/output#2019-07-3115:33tony.kaymakes sense, I just assumed the EQL notation was what was always used in output#2019-07-3115:56wilkerlucioyeah, that's a good point to clarify in the docs, and we can probably support the union syntax as well, a simple impl could flat it out at definition time#2019-07-3115:57wilkerlucioI'm taking notes on things I want to improve on the new docs (that are in progress as of now), that's surely a good one to add#2019-08-0213:34souenzzoeql specify things like [(:foo/bar {:default "---"})] but datomic expect [(:foo/bar :default "---")] There is some standard solution to this? I'm developing a custom ast->datomic-selector ATM#2019-08-0216:47wilkerlucio@souenzzo is [(:foo/bar :default "---")] some sort of new syntax? as far as I remember datomic doens't have params in any way, they do have something called attributes with options (https://docs.datomic.com/on-prem/pull.html#attribute-with-options) but the syntax is [:foo/bar :default "---"] (no parentesis), is this what you mean?#2019-08-0216:48wilkerluciooh. reading again I'm seeing that (weird, I though that was vector only, but ok)#2019-08-0216:48wilkerluciono standard way to do it in terms of pathom or eql#2019-08-0216:49souenzzointerested in ast->datomic-selector inside eql ? I think that it could be a external lib#2019-08-0217:18wilkerlucioI think its not in the context, eql is supposed to be very clean and raw, this is more an extension/integration feature, better to have it as a separated thing#2019-08-0217:18wilkerlucioor maybe some other package, eql-tools? not sure about it, what you think?#2019-08-0217:20souenzzoi think that eql-tools or eql-datomic is a better idea once it will keep eql "core" smaller#2019-08-0302:23souenzzothere is examples of connect + unions? docs just cover "internal" stuff#2019-08-0302:42souenzzo
(let [env {::pc/register (pc/resolver `a {::pc/output [{:a {:b [:c]}}]} (fn [& _] {:a {:b {:c 42}}}))}
      ctx {::p/reader  [p/map-reader pc/all-readers]
           ::p/plugins [(pc/connect-plugin env)]}
      parser (p/parser ctx)]
  (parser ctx [{:a {:b [:c]}}]))
=> {:a {:c :com.wsscode.pathom.core/not-found}}
;; expected: {:a {:c 42}}
#2019-08-0313:05wilkerluciowe talked about that recently here: https://clojurians.slack.com/archives/C87NB2CFN/p1564583323173600#2019-08-0313:06wilkerlucioI agree, we need better docs around that too, I'm doing a revamp on the docs now, I'll add something on thta#2019-08-0511:17Andreas EdvardssonI’m trying to get a query to be read-consistent by providing a ā€œdb as a valueā€ in the env, which works fine as long as I provide it myself. However, if no db value is provided, I have attempted to use a plugin to populate the environment with the value in question, which almost works. However, not fully understanding pathom yet, I suspect that this breaks with parallel resolving (or something like that) within a single query. Any tips on how to best implement this pattern? Thanks for providing a super cool library!#2019-08-0513:48wilkerlucio@andreas179 hello, for your use case I think :wrap-read is not a good option, you can instead use a :wrap-parser to work around the whole processing, so you can set the db once and get that shared in env across the readers, very similar to what you did#2019-08-0513:58Andreas EdvardssonAh, alright! Sounds exactly like what I am after! I find the sheer amount of flexibility and extensibility Pathom provides just astonishing, even though the docs takes a while to digest :) Thanks @wilkerlucio !#2019-08-0514:02wilkerluciothanks for the feedback, I'm currently changing many things about the docs, I hope the new version can be easier to digest šŸ˜‰#2019-08-0616:59souenzzo
(let [register [(pc/mutation `inc {} (fn anc [{::keys [state]} _]
                                       ;; how to update the `st` value?
                                       (swap! state inc)
                                       {}))
                (pc/resolver `v {::pc/output [:v]} (fn [{::keys [st]} _] {:v st}))]
      ctx {::p/reader  [p/map-reader
                        pc/parallel-reader]
           ::p/plugins [(pc/connect-plugin {::pc/register register})]
           ::p/mutate  pc/mutate-async}
      p (p/parallel-parser ctx)
      state (atom 0)]
  (async/<!! (p (assoc ctx ::state state
                           ::st @state)
                `[:v
                  {(inc) [:v]}]))
  (comment
    =expected=>
    {:v               0
     clojure.core/inc {:v 1}}
    =actual=>
    {:v               0
     clojure.core/inc {:v 0}}))
Is possible to update a value in context after a mutation? (just for it's children )
#2019-08-0617:37wilkerluciocan you elaborate the use case? there are some options to give data down, the simplest is just put the data in the mutation response, then they will be part of the input#2019-08-0617:50souenzzoI'm planning to use the "db value" from #datomic ATM I'm still (-> ctx :conn d/db), so every resolver use a "new" db#2019-08-0618:03wilkerluciohumm, there is a feature to change env during processing, that is returning ::p/env, but I just tried with mutations and something is broken there#2019-08-0618:04wilkerlucioit would be something like this:#2019-08-0618:04wilkerlucio
(quick-parser {::pc/register [(pc/mutation 'mut
                                  {}
                                  (fn [env _]
                                    {::p/env (assoc env :env-data 42)}))

                                (pc/resolver 'from-env
                                  {::pc/output [:env-data]}
                                  (fn [{:keys [env-data]} _]
                                    {:env-data env-data}))]}
    '[{(mut)
       [:env-data]}])
#2019-08-0618:04wilkerlucioI'll try to get that to work, not sure whats breaking about it#2019-08-0618:34mdhaney@souenzzo I just faced this recently. Ideally we want to a) use the same version of the db with each resolver and b) update that db after a transaction, so mutation joins are pulling against the new db. What I came up with is to have a global resolver that inserts the db into the response. Query resolvers have the db as a required input when needed, and it gets inserted by the first resolver that needs it (by the global resolver) and then subsequent resolvers reuse it, so a) is satisfied. For mutations joins, they simply return the updated db along with whatever you are returning to establish context for the query, and then this db is used for subsequent resolvers to resolve the query, satisfying b). I hope that explanation is clear. It’s actually very little code and easy to understand once you see it. If it’s not clear, I can post a gist in a bit as an example.#2019-08-0620:33souenzzoseems like a simpler solution
(let [register [(pc/mutation `inc {::pc/output [::st]}
                             (fn anc [{::keys [state]} _]
                               {::st (swap! state inc)}))
                (pc/resolver `st {::pc/output [::st]}
                             (fn [{::keys [state]} _] {::st @state}))
                (pc/resolver `v {::pc/input  #{::st}
                                 ::pc/output [:v]}
                             (fn [_ {::keys [st]}]
                               {:v (str "v: " st)}))]
      state (atom 0)
      ctx {::p/reader  [p/map-reader
                        pc/parallel-reader]
           ::state     state
           ::p/plugins [(pc/connect-plugin {::pc/register register})]
           ::p/mutate  pc/mutate-async}
      p (p/parallel-parser ctx)]
  (async/<!! (p (assoc-in ctx [::p/entity ::st] @state)
                `[:v
                  {(inc) [:v]}])))
#2019-08-1215:46cjsauer@mdhaney that’s a really cool solution. Db-as-a-value continues to have so many interesting consequences. In this case, the database is no longer part of the environment, but something directly resolvable...it’s not just ā€œcoolā€, but totally natural. Thanks for sharing. #2019-08-0618:35wilkerlucio@mdhaney that's a good approach I think, you can already leverage caching with it, and can override if you want to, well done šŸ‘#2019-08-0618:46mdhaney@wilkerlucio thanks. šŸ™‚. I would love to get your feedback on another approach I’m using with Datomic (Peer, not Cloud, which probably makes a difference). Originally, to pull say all the local attributes (non-refs) on an entity, I was just doing it in a single resolver with one big pull expression to Datomic. Easy enough, but seemed somewhat inefficient to me (what if you only need 1 attribute but you’re always pulling 20-30?). What I’ve started doing instead is using the Datomic entity api, so in my resolvers that resolve idents, I grab the entity and shove it in the output. Then I have a separate resolver for each attribute that pulls the value off the stored entity (which sounds tedious, but is easily wrapped in a macro to make it usually a 1 liner, unless you need to do more tweaking of the value). I haven’t benchmarked it yet to compare, but this seems more efficient to me. Not only are you only pulling just the attributes you need (because the Datomic entity api lazily resolves the attribute values) but also the separate resolvers for each attribute are run in parallel, as opposed to waiting for the entire pull query to finish. Just curious if you had experimented with this approach or not.#2019-08-0618:49wilkerlucio@mdhaney I would love to see some benchmarks, I would assume for peer it doens't matter as much since the DB is mostly on memory, so reading a single property vs many should not make a different as far as I understand (but benchmarks can prove me wrong :)). on the using entities idea, I think its valid as well, you could write a new reader similar to the map-reader and use a different key on the env to read from it, so you can cascade down to map-reader when you don't have a datomic entity to pull from, makes sense?#2019-08-0619:25mdhaneyInteresting, I’ll have to look into the custom reader.#2019-08-1023:41Vincent Cantinhi @wilkerlucio, thank you for sharing pathom, that’s a great library, with great ideas.#2019-08-1023:42Vincent CantinI started to read the whole documentation and noticed some typos here and there. I am going to provide a PR at some point.#2019-08-1023:43wilkerlucio@vincent.cantin hello, thanks for looking it up, if you like to help with docs typos, I'm currently rewriting a lot of parts of docs, so master is quite outdated about it, but you can see the new things progress at: https://github.com/wilkerlucio/pathom/tree/docs-v2#2019-08-1023:43wilkerlucioso if you like to collaborate, please do on top of that#2019-08-1023:45wilkerlucioah, worth mention, new docs will live in: https://github.com/wilkerlucio/pathom/tree/docs-v2/docs-src#2019-08-1023:45wilkerluciomoving from a single file things to many files, and using http://antora.org to generate the output#2019-08-1023:54Vincent CantinMaybe that’s something that should be written on the README.md on the master, so new comers who like to stay on the edge can go to read the doc-v2 directly, while knowing that it is a WIP - reading the documentation is a long process, people usually don’t want to do it twice.#2019-08-1023:57wilkerlucioyeha, its just I'm doing some re-organization as well, the risk of moving then there now is looking at things that may not make sense, given that, the new docs will be way more friendly for peoplo to edit, with buttons to edit each page in the output, I hope I get those out soon, but you know, there always this time thing#2019-08-1023:58Vincent CantinI will try to help.#2019-08-1103:35Vincent Cantin@wilkerlucio I am not sure how to build the documentation on the docs-v2 branch. I still need to read a full documentation to learn the library, so for now I am going to continue reading from https://wilkerlucio.github.io/pathom/ and when I see some typos, I will find and fix them in docs-v2.#2019-08-1103:38wilkerlucio@vincent.cantin oh, let me make that a little easier#2019-08-1103:38wilkerlucioso, first you need to install antora: https://docs.antora.org/antora/2.0/install/install-antora/#install-antora-globally-using-npm#2019-08-1103:39wilkerluciothen run: antora docs-dev.yml#2019-08-1103:39wilkerlucioplease let me know if that works for you#2019-08-1104:17Vincent CantinIt works, I was trying with the wrong yml file.#2019-08-1103:40wilkerlucio(just be sure to be on latest, I modified docs-dev.yml to be simpler, the previous version required a local copy of the ui bundle to be present, the latest pulls from a remote source)#2019-08-1103:42wilkerlucioah, also you need to compile the book to get the examples: shadow-cljs watch book#2019-08-1103:42wilkerlucioyou need to run antora compilation after the build is done to get it there#2019-08-1103:44wilkerlucioif you keep the shadow compilation running you can use http://localhost:8082 to see the new docs#2019-08-1600:42kennyAre there any examples of using pathom with Datomic? I'm curious about the various approaches you could take.#2019-08-1603:40wilkerluciohello Kenny, welcome, so far no docs yet, but I plan to add those since there is a lot of people that ask. I have never used pathom with datomic myself, but what most people are doing is writing one resolver for each "entity" that returns the scalar values of it, while relationships usually get their own resolver, that makes sense?#2019-08-1614:57kennyYep! Going for a POC within the next week. If each entity has their own resolver, is each resolver parameterized, somehow, to only pull the necessary attributes from Datomic?#2019-08-1615:01wilkerlucioit could be done, but to be honest it seems like fetching those extra things on the pull is not a problem (specially if you are not running on datomic cloud). for the user output EQL will select only the request keys, so the extra on the pull will live only during processing time (so they can used to process other dependencies, but unless the user asks for then, they will not appear in the output)#2019-08-1615:03kennyYeah, generally seems pretty low cost. Maybe in the same realm, if you have several nested entities and a resolver per entity, won't that result in several Datomic pull's? It seems more efficient to just do the pull at the root.#2019-08-1618:02wilkerlucioyeah, in the ideal world we could detect what can be resolved from the query and delegate as much as possible to pull and send that subquery there, and later process the rest. but since we don't have that integration (GraphQL integration does a similar thing, but there is nothing for Datomic at this time) its about trying out and balacing according to your use case. makes sense?#2019-08-1618:41kennyRight, makes sense.#2019-08-1603:53wilkerlucio#2019-08-1613:36eoliphantOba!#2019-08-1613:35Vincent CantinI have a question: how would you do to map resolvers to APIs which require multiple input parameters, with those parameter's numericity been one-to-many.. For example, imagine a legacy API designed like that: GET /api/{blogId}/{articleId}/{versionId}/md#2019-08-1613:35kszabofor these kind of scenarios I just use the fact that you can use rich EDN for idents#2019-08-1613:36kszaboi.e: {[:my-legacy-req-map {:blog-id 1 :article-id 2 :version-id 4}] [:foo :bar]}#2019-08-1613:37kszaboessentially you create an entity with composite keys via maps#2019-08-1613:38kszaboand the resolvers can map to your regular old :blog/id :article/id :version/id entities allowing graph traversal#2019-08-1613:39Vincent CantinI think that it would work, but I don't like seeing the bad design pattern of the API leaking into the queries. The user of pathom is supposed not to remember how the API is structured, ideally.#2019-08-1613:39Vincent CantinI would be happy if there was a solution involving only the resolvers.#2019-08-1613:42kszaboIf you have a datasource which requires 3 things to look up something, you can’t express it without any less. The only alternative I see is via using :pathom/context#2019-08-1613:42Vincent Cantinor ... maybe the {:blog-id 1 :article-id 2 :version-id 4} data could be the output of a resolver ..#2019-08-1613:42kszabothat way you could define a resolver which requires the ids of the 3 above and output whatever is under /md for that resource#2019-08-1613:43Vincent Cantinlike that :blog-id -> :blog-article-id -> :blog-article-version-id ?#2019-08-1613:43kszabo
[{([:github.repository/name "pathom"] {:pathom/context {:github.repository/owner "wilkerlucio"}}) [...]}]
#2019-08-1613:43kszabothis is essentially the same problem#2019-08-1613:43kszabohttps://wilkerlucio.github.io/pathom/#MultiInputIdents#2019-08-1613:44eoliphantthe existing parameter support should work fine no? [(::blog {:id .. :article .. :version ..) ...])#2019-08-1613:45eoliphantah but I see the issue in terms of idents#2019-08-1613:59Vincent CantinI think my solution would work, but I did not test it yet. It would look like that .. Resolvers: - "get-blog-articles" [:blog-id] -> [:article-id :blog-article-id] - "get-article-versions" [:blog-article-id] -> [:version-id :blog-article-version-id] - "get-version-markdown" [:blog-article-version-id] -> [:markdown] A query could then be as simple as: [{:blog-id [{:article-id [{:version-id [:markdown]}]}]}#2019-08-1614:01Vincent CantinDo you think it would work?#2019-08-1614:03Vincent CantinThe trick would be to have a 1-to-1 relation with some data and the path which was used to reach it, and let pathom use it when it is needed.#2019-08-1614:49wilkerlucio@vincent.cantin if you dealing with an API that requires the whole path to reach some entity, I would consider that a composite key, so a composed ident can work, as: [:article/id ["blog-id" "article-id" "version"]], the solution you proposed can work, but adds "nesting requirements" and can make hard to point directly to the resource, if you use something like Fulcro it really helps if you have a global atomic identifier for everything so it can be used as reference for that resource. You can also get creative about how to generate this, you can have processes to figure out other parts (if makes sense and the API provides support, for example, you could have something without the article version that resolves the latest and fills up the version, returning the final article ident complete)#2019-08-1614:50wilkerluciomakes sense?#2019-08-1614:51Vincent CantinYes, that helps. You are right, the nesting requirement is not good.#2019-08-1715:49Vincent CantinI cannot find the documentation (not related to GraphQL) for aliasing a keyword to another one, in my case session/room-id to room/id but I remember seeing something about that in some videos.#2019-08-1715:51Vincent CantinOh, I just found it, never mind. https://cljdoc.org/d/com.wsscode/pathom/2.2.23/api/com.wsscode.pathom.connect#alias-resolver#2019-08-1716:12Vincent Cantin@wilkerlucio I think that pc/alias-resolver should be mentioned in the basic part of the documentation of resolvers, with a small example. Users need to be aware of its existence early as they will need to use it early too.#2019-08-1716:13Vincent CantinThe video you made mentioned about it, but it is absent in the guide.#2019-08-1915:55Vincent CantinI started to use pathom and promote it around me. I made a basic demo during a conference last weekend: https://github.com/green-coder/try-pathom/tree/master/src/try_pathom/coscup#2019-08-2113:42wilkerlucioabout these resolvers here: https://github.com/green-coder/try-pathom/blob/master/src/try_pathom/coscup/resolver.clj#L62-L75#2019-08-2113:43wilkerluciosince they provide nested things (both :english-sessions :room/session-ids are nto scalars), the output should have the full shape of it#2019-08-2113:43wilkerlucioalso, I guess :room/sessions would be a better name, makes sense?#2019-08-2102:24wilkerlucio@vincent.cantin more docs updates in the works, including new section for alias resolvers: https://github.com/wilkerlucio/pathom/pull/116/files#diff-02187d2cad82ef844c591fbaf4639f35R259-R282#2019-08-2122:24souenzzoquestion about EQL there is a simpler way to do this: (update ast :children conj (eql/expr->ast :com.wsscode.pathom/trace)) ?#2019-08-2123:14wilkerlucio@souenzzo yes, but use eql/query->ast1 instead of expr->ast#2019-08-2712:51souenzzoWhy use query->ast1 ? :foo isn't a query, or is?#2019-08-2712:35kszaboI might be missing something trivial, but in my ::p/process-error handler I can’t see the full tx in the env to report/log#2019-08-2714:10wilkerluciohumm, erros go localized, so the AST will be at the location it failed, maybe that's a missing feature (having access to the root query)#2019-08-2714:10wilkerluciolet me double check if I didn't add that#2019-08-2714:16kszaboI checked with REBL and there was no such thing in the env, using quick-parser#2019-08-2714:16kszaboI essentially circumvented it with this:
(defn parse-fn! [parser]
  (fn [env query]
    (-> (parser (assoc env ::p/tx query) query)
        (async-clj/<!!maybe)
        (async-clj/throw-err))))
#2019-08-2714:17kszabobut of course this has a lot of pitfalls#2019-08-2714:20kszaboI expected it to be implemented here: https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/core.cljc#L969#2019-08-2714:20wilkerlucioit sets the parent query, but that changes on each path going down#2019-08-2714:20wilkerlucioseems like a simple thing to add (you can also do it as a plugin)#2019-08-2714:21wilkerlucioI think pathom doesn't provide that, but I'm willing to add, can you please open an issue for it?#2019-08-2714:33kszabosure#2019-08-2714:34kszabocan I ask, will it be ::p/tx?#2019-08-2714:34kszaboor would you decide that later#2019-08-2714:34kszabojust so that I can assume in my code that it will be a some key#2019-08-2714:34kszabowithout breakage when I upgrade and remove my solution#2019-08-2714:35kszabohttps://github.com/wilkerlucio/pathom/issues/118#2019-08-2716:38wilkerlucio@U08E8UGF7 I think ::p/root-query will be the name, makes sense?#2019-08-2716:39wilkerlucioso it aligns with current ones ::p/query and ::p/parent-query#2019-08-2716:48kszabosure, sounds good#2019-08-2716:48kszabothanks for being so responsive! šŸ·#2019-08-2719:06kennyCan you require that a particular key exists in the env for a resolver to get called?#2019-08-2719:09wilkerluciono, that's inputs job, can you tell more about the use case?#2019-08-2719:11wilkerlucioone simple way to handle it, is making a resolver that pulls your key from env to some output, and them depend on that, makes sense?#2019-08-2719:11wilkerlucio(if the key is not present on env, return a blank map, so pathom understands thats unreachable)#2019-08-2719:12kennySure. We have a Ring application that has an eql endpoint. Certain requests are authorized to access different dbs. When a query tries to pull data from a db that is does not have access to, it should return not-found. That exactly what I ended up doing šŸ™‚ Seems like an acceptable approach.#2019-08-2719:19wilkerluciothe only downside I see he is that you enable users to access that directly (and you may not want that), to go around that you can have a plugin that removes this from the output#2019-08-2719:19kennyAccess the resolver?#2019-08-2719:20wilkerlucioaccess the db in your case#2019-08-2719:21kennyIn this case I wrapped the resolver fn in a when so the whole thing is skipped. I imagine more complex use cases may require something like what you just described.#2019-08-2719:23wilkerluciocool, yeah, you don't have to acutally put the value there, just a signal its available šŸ™‚#2019-08-2719:16kennyIs there a spec for an EQL query?#2019-08-2719:18kennyAh, part of edn-query-language: :edn-query-language.core/query#2019-08-2719:18kennyThe generator is not very fast 😬#2019-08-2719:19wilkerlucioyou can also see it here: https://edn-query-language.org/eql/1.0.0/clojure-specs.html#2019-08-2719:20wilkerlucioI recommend you avoid using the "raw generator", that will generate too much randomness, usually is not what you want#2019-08-2719:20wilkerluciobut I designed them in a way that you can "cherry pick" how the generation works, you can find info for it here: https://edn-query-language.org/eql/1.0.0/library.html#_generation#2019-08-2719:21kennyOh I see. They have a little make-gen helper. Why didn't they make that the default gen for the specs?#2019-08-2719:21wilkerluciothis was a way I found to make then very customizable#2019-08-2719:22wilkerlucioplease check the docs first, the way they are done allow you to customize every detail on how queries are generted, since each user may have different cases (in my own usages I have several different configurations, depending on what I want to test)#2019-08-2719:22kennyIt seems fair to override the default generator for the specs so it at least generates in a timely manner.#2019-08-2719:22kenny(gen/generate (s/gen ::eql/query)) will hang the repl after a couple runs.#2019-08-2719:23wilkerlucioyou can reduce the :size if you like to avoid that#2019-08-2719:23wilkerluciobut I highly recommend you make a custom setup, its pretty fun, and gets more precisely what you need#2019-08-2719:24wilkerlucioends up looking something like this:#2019-08-2719:24kennyI absolutely agree. Just suggesting a better default šŸ™‚#2019-08-2719:24wilkerlucio
(gen/sample (eql/make-gen {::eql/gen-property ; 
                       (fn [_] (gen/elements [:id :name :title :foo :bar]))}
              ::eql/gen-query) 
#2019-08-2719:24wilkerluciošŸ™‚#2019-08-2719:24wilkerluciothe default will do a max of 5 depth levels#2019-08-2719:24wilkerlucioI don't remember exactly makes that hangs on the default#2019-08-2719:24wilkerluciothere just too many moving pieces#2019-08-2719:27wilkerluciosince we are talking generators, another cool feature you may want to look at is the connect generators, using that you can limit the query generation to your own index (so it only generates things your parser can resolve)#2019-08-2719:27wilkerluciorun this fn with your indexes and you get it: https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/connect/gen.cljc#L80#2019-08-2719:28kennyOoo, very cool. That's exactly what my end goal was!#2019-08-2800:47kennyIs there a way to customize when :com.wsscode.pathom.core/not-found shows up? For example, say I have an attribute :user/games which is cardinality many of maps. Each :game inside the list has a :game/type. My application want to pull all the :user/games, only pulling particular :game attributes depending on the :game/type, similar to what clojure.spec/multi-spec does. Is there a way to do this?#2019-08-2811:25wilkerlucioabout not-found, what people usually do is set a plugin to remove those from the output, they are internally useful to mark that something can't be realised and I decide to let if flow to the user, because its more information then nil or not having the key, but you can use the plugin: p/elide-special-outputs-plugin, this will remove not-found and reader-errors#2019-08-2811:25wilkerlucioabout the type thing, EQL handles that with Union Queries: https://edn-query-language.org/eql/1.0.0/specification.html#_unions#2019-08-2811:27wilkerluciosome pathom docs on unions: https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/core/entities.html#_union_queries#2019-08-2811:47wilkerluciops: this is a bit outdated, was before connect, with connect the "branch" is usually decided by checking the presence of a key in the entity#2019-08-2815:41kennyOh cool! It sounds like I just need to structure the query correctly and Connect will figure it all out!#2019-08-2815:49kennyThe p/elide-not-found doc says: > Convert all ::p/not-found values of maps to nil It will actually remove ::p/not-found values, not set them to nil.#2019-08-2818:49wilkerluciocorrect#2019-08-2818:48pserrano@wilkerlucio will there be a talk about pathom in clojure south? Are you going to be there?#2019-08-2818:49wilkerluciohello, I'll be at the event, but I have no talk in this one#2019-08-2818:49wilkerlucioare you coming for the conference?#2019-08-2818:56pserranoSure. Me and @U2J4FRT2T are going to be there#2019-08-2818:57pserranoWe weren't brave enought to try talking about developing react native apps with fulcro + pathom hahahhaha#2019-08-2819:32refsetthere's a good chance that @U4L16CHT9 will mention @U2J4FRT2T's https://github.com/souenzzo/graph-demo in his talk https://clojure-south.com/datomic-vs-crux-and-why-it-matters/ šŸ™‚#2019-08-2821:40wilkerluciolets all meet up there! šŸ˜„#2019-08-3111:44pserranoWe're already here hahaha#2019-08-2821:48wilkerluciojust released Pathom 2.2.24, this release adds support for ::p/root-query @thenonameguy#2019-08-2919:36Ian Fernandezhey guys, I'm using this
(def userCredentialsResp [:attributes
                          :blockedUsers
                          :category
                          :citizenId
                          :city
                          :countryCode
                          :diary
                          :email
                          :firebaseUserToken
                          :hourly
                          :industryList
                          {:languageList [:languageId
                                          :proficiency]}
                          {:skillList [:label
                                       :skillId]}
                          :name
                          :password
                          :phone
                          :positionId
                          :profileImage
                          :stateId
                          :summary
                          :title
                          :userId])
 (pcgql/query->graphql `[{(inputCredentials {:userId "6262818231812096​"
                                                        :name "DV"
                                                        :title "C"
                                                        :category "technology"
                                                        :phone "+55660"})
                                     ~userCredentialsResp}] {})
#2019-08-2919:36Ian Fernandezand its's returning me a query with this character \\u200b#2019-08-2919:36Ian Fernandezlike this:
"mutation {\n  inputCredentials(category: \"technology\", phone: \"+55660\", name: \"DV\", title: \"C\", userId: \"6262818231812096\\u200b\") {\n    attributes\n    blockedUsers\n    category\n    citizenId\n    city\n    countryCode\n    diary\n    email\n    firebaseUserToken\n    hourly\n    industryList\n    languageList {\n      languageId\n      proficiency\n    }\n    skillList {\n      label\n      skillId\n    }\n    name\n    password\n    phone\n    positionId\n    profileImage\n    stateId\n    summary\n    title\n    userId\n  }\n}\n"
#2019-08-2919:55wilkerlucioI can't think of any reason why pathom would add that, can you double check if this string doesn't have some hidden character? maybe rewrite just that string and check#2019-08-2919:57Ian FernandezI've rewrote it many times =< dunno why... for now I've just used #(clojure.string/replace % #"\\u200b" "")#2019-08-2919:58Ian Fernandezit's very strange, I'll see if another case will appear, thanks#2019-08-2920:00wilkerlucioif you can get a consistent repro of the bug, please open an issue so I can look later#2019-08-2920:00Ian Fernandezthanks!#2019-08-2919:37Ian Fernandezanyone knows why this character is appearing?#2019-08-2919:43mssis there a way to elide the mutation name from the parse result using the mutate-async mutation fn? is that even advisable? trying to get pathom to hook into fulcro 3's error handling mechanism and trigger an error-action for my mutation. wondering what peoples’ setups look like for that#2019-08-2919:48wilkerluciothis should work as same as in F2, its not desirable to remove it from the results, Fulcro expects the error details to be inside of it#2019-08-2919:52mssI hear that. I guess what I’m struggling with is that the error is boxed up inside of a map like {my-mutation-name {::pc/reader-error ...}}. wondering if there’s any easier way on the api handler side or on the f3 side to determine that it’s an error from the parser besides doing something like (contains? (first (vals parse-result)) ::pathom/reader-error)#2019-08-2919:56msshttps://wilkerlucio.github.io/pathom/#_error_handling#2019-08-2919:57mssper the docs here it seems like there should be a ::p/errors key at the top level of the parse result map, but that doesn’t seem to be the case for an error thrown by ex-info. maybe I’m misreading or missing something in my env beyond the error-handler-plugin?#2019-08-2920:05wilkerlucioyou need to set ::p/process-error in your environment, this is a fn that receives [env error], the return of it will be the value there in case of errors#2019-08-2920:05wilkerluciomakes sense?#2019-08-2920:06mssyep, absolutely. really appreciate the help and all the work you put into this. really a phenomenal library#2019-08-2920:28kennyDoes ::pc/params do anything?#2019-08-2920:32wilkerluciocurrently its documentation only (it shows up in the Index Explorer). but the name is official and consistent, so you can implement things on top of that (maybe a plugin that does clojure.spec validations using it?), Pathom may add features on top of that, but right now is just official name and doc#2019-08-3018:41kennyI am getting an error from a missing mutation:
#:integration{retract "class clojure.lang.ExceptionInfo: Mutation not found - {:mutation integration/retract}"}
What would be removing the pathom errors key from the parsers return value?
#2019-08-3018:42kennyActually there doesn't seem to be an error key for this?
(def example-parser
    (p/parser
      {::p/env     {::p/reader               [p/map-reader
                                              pc/reader2
                                              pc/open-ident-reader
                                              p/env-placeholder-reader]
                    ::p/placeholder-prefixes #{">"}}
       ::p/mutate  pc/mutate
       ::p/plugins [(pc/connect-plugin {::pc/register []})
                    p/error-handler-plugin
                    ;; removes ::p/not-found from outputs
                    (p/post-process-parser-plugin p/elide-not-found)
                    p/request-cache-plugin
                    p/trace-plugin]}))

(example-parser {} `[(foo/bar {})])
#:foo{bar #:com.wsscode.pathom.core{:reader-error "class clojure.lang.ExceptionInfo: Mutation not found - {:mutation foo/bar}"}}
#2019-08-3018:49kennyWhat is the first arg to ::p/process-error fn?#2019-08-3019:13kennyYeah, when a mutation throws an exception, no ::p/errors key is attached to the return.#2019-08-3020:19wilkerlucio@kenny hello, this code is the full example? the mutation seems to be really missing there, also I'm not seeing a configuration for the ::p/process-error#2019-08-3020:19wilkerluciowhat you are seeing is just the default error implementation#2019-08-3020:20wilkerlucioaltough is not the best, I don't wanna change because that would mean break the interface for current users that for any reasons didn't need to change it#2019-08-3020:26kenny@wilkerlucio Yes this is the full example. When an error occurs with a non-mutation resolver, you get the ::p/errors get on the top level map. This lets higher level code decide what to do with the response (e.g. if an error occurred perhaps send a 500 HTTP response to the client).#2019-08-3020:31wilkerlucio@kenny this goes a bit against the idea of soft fails, Pathom embraces that if one thing fail, others can still succeed, so we don't encourage this, but if you think makes sense for you, you can do via plugin, you can detect mutations (symbols in the root map) and check on that as well (that + root), makes sense?#2019-08-3020:35kenny@wilkerlucio I suppose I could do that. The asymmetry here is a bit confusing though. For example:
(pc/defmutation example-mutation
  [_ _]
  {::pc/sym    'example
   ::pc/input  []
   ::pc/output []}
  (throw (ex-info "uh oh" {})))

(pc/defresolver example-resolver-throwing
  [_ _]
  {::pc/output [:foo]}
  (throw (ex-info "uh oh" {})))

(def example-parser
  (p/parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/reader2
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate
     ::p/plugins [(pc/connect-plugin {::pc/register [example-mutation
                                                     example-resolver-throwing]})
                  p/error-handler-plugin
                  p/request-cache-plugin
                  p/trace-plugin]}))
Calling the resolver:
(example-parser
    {}
    [:foo])
=> {:foo :com.wsscode.pathom.core/reader-error,
 :com.wsscode.pathom.core/errors {[:foo] "class clojure.lang.ExceptionInfo: uh oh - {}"}}
Calling the mutation:
(example-parser
    {}
    '[(example {})])
=> {example #:com.wsscode.pathom.core{:reader-error "class clojure.lang.ExceptionInfo: uh oh - {}"}}
#2019-08-3020:35kennyIt's strange that both mutations and resolvers handle errors differently.#2019-08-3020:36kennyI would've expected the mutation call to attach ::p/errors just like the resolver call did.#2019-08-3020:38wilkerlucio@kenny yeah, I can understand the issue with the inconsistency there, to be honest, these days I do the opposite, I move the errors from the outside to the inside (makes much easier to report problems in the UI), pathom has a specific helper for that: https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/core.cljc#L825-L846#2019-08-3020:38wilkerluciothe reason this ended up happening was historical, I kind copy over what GraphQL did in terms of errors (having a separated branch)#2019-08-3020:39wilkerluciobut after experience doing it I feel like its better to have then in the same level they happen#2019-08-3020:47kennyIt's definitely a bit strange because typically you want the HTTP status code to match what happened with request processing.#2019-08-3020:57kenny@wilkerlucio Is there a way to change the format of errors? e.g. I want them to be in a cognitect anomaly.#2019-08-3021:05wilkerlucio@kenny yes, use the ::p/process-error in your env#2019-08-3021:06wilkerluciothen you have full control about how they get exposed#2019-08-3021:06kennyOh cool! Pathom won't care that I messed with those things?#2019-08-3021:06wilkerluciohttps://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/core/error-handling.html#2019-08-3021:06wilkerluciono, that's expected#2019-08-3021:06wilkerlucioI mean, this how to format the error output, the position will remain#2019-08-3021:13kennyWhat is the first arg to ::p/process-error fn?#2019-09-0104:07wilkerluciothe env#2019-09-0423:07kennyIs there a way to have ::pc/input be parameterized based on another keyword? e.g. When :type is :a, I require :a1. When :type is :b, I require :b1.#2019-09-0518:24kenny@U066U8JQJ Is there a way to do this?#2019-09-0518:40wilkerluciono, this sounds like a case for unions though#2019-09-0518:40wilkerluciohad you checked those?#2019-09-0519:42kennyCan you use unions for ::pc/input?#2019-09-0520:34wilkerluciono, that's more a thing you have to do at query level, when you have different queries depending on the branching type#2019-09-0520:34wilkerluciothis not a resolver level concern#2019-09-0520:35wilkerluciowhats your use case?#2019-09-0520:46kennyExactly what I described above šŸ™‚#2019-09-0520:47kennyI have a single map that requires different ::pc/inputs depending on the value of a key in the map.#2019-09-0521:01wilkerluciothat's the implementation detail you are looking for, but what is your use case? what kind of query you need to handle?#2019-09-0521:03wilkerluciojust to be clear, what you asked specifically, is not supported, there are some already complicated rules to ensure everything runs as is, this is not a trivial thing to change, but there are many ways to model things, so I'm tryign to get your problem so we can try to model it in a way that aligns with how the library works#2019-09-0521:04kennyI have an entity that looks something like this
{:metric/integration            {:integration/type :integration.type/aws}
 :metric/source                 :metric.source/aws-cloudwatch
 :metric.cloudwatch/metric-name "Foo"}
We have a function called metric-query that will take that metric map and issue a query to the correct destination based on attributes defined in the map. That function will return a vector of points that I want to attach to this map under the key :metric/provider-points.
#2019-09-0522:22wilkerluciosorry, I'm not getting it yet, can you tell me some query examples and expected results?#2019-09-0600:36kennyI actually think I have another way of doing this.#2019-09-0501:19kennyIsn't this supposed to place errors on the same level as the attribute? Perhaps I am missing something about how this is supposed to be used.
(p/raise-errors {:metric/provider-points :com.wsscode.pathom.core/reader-error,
                 :com.wsscode.pathom.core/errors
                                         {[:metric/provider-points]
                                          {:cognitect.anomalies/category :cognitect.anomalies/incorrect,
                                           :cognitect.anomalies/message  "foo"}}})
=>
{:metric/provider-points :com.wsscode.pathom.core/reader-error,
 :com.wsscode.pathom.core/errors #:metric{:provider-points #:cognitect.anomalies{:category :cognitect.anomalies/incorrect,
                                                                                 :message "foo"}}}
#2019-09-0501:24kennyThis seems more in line with what I expected:
(defn raise-reader-errors
  [parser-output-data]
  (reduce
    (fn [m [path err]]
      (if (= ::p/reader-error (get-in m path))
        (assoc-in m path err)
        m))
    (dissoc parser-output-data :com.wsscode.pathom.core/errors)
    (get parser-output-data :com.wsscode.pathom.core/errors)))
#2019-09-0517:18wilkerlucio@kenny your case is a bit different because its an error in the root, thats interesting case to see, most of the time the queries I use have an ident at the root (so provide some data), in case your error is at the root, then the error position is already correct, makes sense?#2019-09-0600:37kennyget at the path :metric/provider-points does not return the error.#2019-09-0600:15kennyI have this query sent to a resolver.
[{([:integration/id #uuid "81890273-ccee-43cb-88b6-54bce85438f7"]
   {:metric
    {:metric/id   #uuid "26b0037a-8a81-414b-b64d-906a9e607320",
     :metric/name "test"}})
  [{:metric/provider-points [:points]}]}]
Inside the resolver I am printing (-> env :ast :params) and it is nil. Is this expected? Aren't you supposed to be able to pass parameters to resolvers?
#2019-09-0600:35kennySimple example:
(pc/defresolver points-example
  [env
   _]
  {::pc/input  #{:foo/id}
   ::pc/output [:foo/thing]}
  (let [params (p/params env)]
    (prn params)
    {:foo/thing 1}))
When calling my parser:
(parser '[{([:foo/id "abc"] {:sort :asc}) [:foo/thing]}])
{}
=> {[:foo/id "abc"] #:foo{:thing 1}}
The printed params map is simply {} even though I passed in parameters.
#2019-09-0600:39kennyThe query is definitely parsed as having parameters:
(edn-query-language.core/query->ast 
  '[{([:foo/id "abc"] {:sort :asc}) [:foo/thing]}])
=>
{:type :root,
 :children [{:type :join,
             :dispatch-key :foo/id,
             :key [:foo/id "abc"],
             :params {:sort :asc},
             :meta {:line 2, :column 6},
             :query [:foo/thing],
             :children [{:type :prop, :dispatch-key :foo/thing, :key :foo/thing}]}]}
#2019-09-0600:49kennyMaybe that's what this is? https://github.com/wilkerlucio/pathom/issues/93#2019-09-0600:50kennySeems like this sort of thing should definitely be possible.#2019-09-0600:56kennyWrote this
(defn ident-reader
  "Like ident-reader, but ident key doesn't have to be in the index, this will respond
  to any ident join. Also supports extra context with :pathom/context param."
  [env]
  (if-let [key (p/ident-key env)]
    (let [params (get-in env [:ast :params])
          extra-context (:pathom/context params)
          ent           (merge {key (p/ident-value env)} extra-context)]
      (p/join (atom ent) (update-in env [:eql/params key] merge params)))
    ::p/continue))
Seems to work by adding the params for a key into the env. Don't think it's very robust though.
#2019-09-0614:08wilkerlucioyeah, I think its ok for a specific solution, but I'm not confortable as a general one, would be interested to see how that behaves over time, not something I want on core, but fine for users to make things like this šŸ‘#2019-09-0614:52kennyWhy does Pathom not support params for resolvers? That seems so fundamental.#2019-09-0617:22wilkerlucioit does, the problem is just a misunderstanding on how the query works, params are part of the AST, so you can add params for each entry, and the params for the parent join is a different thing, so at AST level, the param is just on the join, not on the attribute, but when you are writing a resolver, the AST is about the attribute, not about the join#2019-09-0617:22wilkerlucioanother way to handle this is using ::p/parent-query, I guess you can get the parent param from there, but see, those are different AST points, so you have to take that in consideration#2019-09-0617:22wilkerluciomakes sense?#2019-09-0617:24wilkerlucioadding: if you add the params to the attribute, then they will show up in params, but params form the join are not on the field, because they are not the same thing#2019-09-0617:25kennyLike this?
'[{[:foo/id "abc"] [(:foo/thing {:sort :asc})]}]
#2019-09-0617:25wilkerlucioyeah, correct#2019-09-0617:25kennyInteresting. That probably makes more sense in general.#2019-09-0617:35wilkerluciothere are some examples demoing the AST with params, can help to understand where things get placed: https://edn-query-language.org/eql/1.0.0/specification.html#_parameters#2019-09-0622:29kennyCan you use joins as ::pc/input?
#{:workload/id
  {:workload/biz-factors [:biz-factor/id
                          :biz-factor/name]}}
Where for my query I just pass in
(parser '[{[:workload/id #uuid "1cf55797-bb5c-49cc-8e71-39a6f59c69b9"]
           [:workload/foo]}])
It appears you can't but maybe I'm doing something wrong?
#2019-09-0703:33wilkerluciono, that's currently not supported, but its something I like to support in the future, for now what you can do to go around this is do a call to the parser from the inside of your resolver, so make the input just #{:workload/id :workflow/biz-factors}, then inside the resolver you can run: (parser env [{:workload/biz-factors [:biz-factor/id :biz-factor/name]}]) (`parser` can be extracted from the env)#2019-09-0703:34wilkerlucionote: if you are using the parallel parser, the return of parser will be a channel, you need to read from it to get the result map#2019-09-0703:35wilkerluciobut depending on how is your resolver for biz-factors, you may not need any of this, if there is only one entry for it, and you don't need something inside to be computed, you can just require the biz-factors and read from it#2019-09-0703:35wilkerluciomakes sense?#2019-09-0900:38kennyYep. Thanks!#2019-09-1116:36cjsauerIs it possible/easy to use Pathom Viz when not using Fulcro?#2019-09-1116:38kszaboyes, there is a standalone branch: https://github.com/wilkerlucio/pathom-viz/pull/7#2019-09-1116:38kszaboI will be working on a bit more sophisticated version available as an nginx served SPA within a docker-image in the near future#2019-09-1116:38kszabofor now you have to build it/deploy it yourself#2019-09-1116:42cjsauerOh cool, thank you for the link. Is a ā€œpathom serverā€ a standardized thing? I imagine it makes assumptions on how the parser is actually exposed via HTTP. This is likely documented somewhere, I may need to RTFM šŸ™‚#2019-09-1116:47cjsauerAh, this is a good hint: https://github.com/wilkerlucio/pathom-viz/blob/b2349e68b011672796c4ed96a8dea7b7bb9491be/src/core/com/wsscode/pathom/viz/standalone.cljs#L25-L30 Looks like the query is encoded in form params the body as transit+json, and the response is sent in the body, also as transit+json#2019-09-1116:48cjsauerI’ve been reading through Fulcro’s networking implementation, and it also seems to use a similar approach (except for maybe sending the query as form-params)#2019-09-1116:50cjsauerOh I may have misunderstood. The ::p.httml/form-params key derives content type from ::content-type https://github.com/wilkerlucio/pathom/blob/9b4c4a6bd07ec5ec570c0596eb446853e2904eed/src/com/wsscode/pathom/diplomat/http.cljc#L35-L37#2019-09-1117:04cjsauerAre these com.wsscode.pathom.diplomat.* namespaces meant to be a general solution to using pathom over the network, or are they mainly intended to be used for development/tooling?#2019-09-1117:14wilkerlucio@cjsauer the diplomat idea is to abrastract away the impl between cljs and clj, this way we can have resolver collections that work on both (just having to replace the HTTP driver)#2019-09-1117:14wilkerlucio@thenonameguy thanks for pointing that out, to be honest I had forgot about it, but just add on my todo list, will try to check and merge it ASAP#2019-09-1117:15cjsauerOhh okay, so the diplomat is meant to be used within resolver implementations? That makes sense.#2019-09-1117:16wilkerlucio@cjsauer here you can find an example: https://github.com/wilkerlucio/pathom-connect-youtube/blob/master/src/core/com/wsscode/pathom/connect/youtube/helpers.cljs#L50-L62#2019-09-1117:17cjsauerI see. And so that resolver would work on both client and server.#2019-09-1117:17wilkerlucioyeah, given you provide a compatible http driver šŸ™‚ (which is a fn that understand the keywords we send)#2019-09-1117:17cjsauerGot it. Thank your for the example Wilker.#2019-09-1217:25kennyIs it ok to have blocking operations in a resolver/mutation when using the parallel parser?#2019-09-1218:02souenzzo@kenny no. It will freeze your system sometimes a dummy solution is wrap your blocking ops with a thread https://github.com/souenzzo/graph-demo/blob/master/src/main/souenzzo/graph_demo/core.clj#L99#2019-09-1218:16kennyHmm. That seems a bit sketchy. #2019-09-1218:27wilkerlucio@kenny if you use the thread pool feature, its ok#2019-09-1218:27wilkerluciothe thread pool thing will run resolvers in a dedicated thread pool, so they wont block the go block threads#2019-09-1218:28wilkerluciohttps://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/thread-pool.html#2019-09-1218:29kenny@wilkerlucio Oh cool! So then you don't need to return a core.async thread from each resolver, right?#2019-09-1218:29wilkerluciocorrect#2019-09-1218:29wilkerlucioeven if you don't use the thread pool, its ok to return strait values#2019-09-1218:29wilkerluciothe thread pool just ensures you are not blocking the go threads#2019-09-1218:30kennyAnd the benefit over the regular parser is that the resolvers are run in parallel, right?#2019-09-1219:05wilkerluciocorrect šŸ‘#2019-09-1219:06wilkerlucio@kenny just a caveat, tune that number, nowadays I believe 200 is too much, at Nubank we are running pretty big instances with 100 and its been fine
#2019-09-1219:06wilkerluciodepending on how much concurrency you are expecting it may be good to bump the Go thread pool as well (core.async default is 8, we are using 32)#2019-09-1219:12kennyThe go thread pool wouldn't need to change if I'm passing a thread pool to Pathom, correct? We don't really use go blocks.#2019-09-1412:40Abhinav SharmaHi guys, could you please tell me how to get the https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/core/async.html#_js_promises example return properly in the repl.#2019-09-1421:02wilkerlucioasync parsers return channels, so to get the value you have to read on it, use go and <! from core async#2019-09-1512:38souenzzo(async/go (prn (async/<! (parser ....))))#2019-09-1521:24Abhinav SharmaAh, got it! Thanks @U066U8JQJ @U2J4FRT2T šŸ‘#2019-09-1513:24bedershey pathomians, I'm trying to contribute to the new documentation, but the Edit Page link is only resolving to a local file. Any idea where to contribute?#2019-09-1519:07wilkerlucio@beders hello, sorry for that, I did a bad deploy yesterday, going to fix a bit#2019-09-1519:08wilkerluciothe docs are mostly here: https://github.com/wilkerlucio/pathom/tree/master/docs-src/modules/ROOT#2019-09-1519:12wilkerlucio@beders docs edit link fixed#2019-09-1519:16bedersthanks!#2019-09-1804:07Abhinav SharmaGuys, excuse me if this has been asked before but I was wondering about something regarding Pathom 1. Is Pathom only meant for fetching data or is it possible to push the data as well? 2. For any API endpoint we need to write the parsers directly against the REST endpoint? Or is there a way to adapt the provided JS lib somehow#2019-09-1809:15kszabo1. Mutations cover this need 2. Yes, generally you write a resolver which fetches your REST response, you usually fully flatten + qualify all attributes according to your own keywords#2019-09-1811:37Abhinav SharmaThanks @U08E8UGF7 for the pointer here 1. I guess that here’s the relevant doc https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/core/mutations.html And https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/connect-mutations.html Do you know of any other example of pushing data to a non-clojure server? 2. Got it šŸ‘#2019-09-1811:41kszabo1. yes, those are the correct docs what do you mean by pushing to non-clojure servers? In the context of Pathom that does not make sense to me#2019-09-1811:42Abhinav SharmaHmm, I was thinking about using Fulcro+Pathom with a firebase backend#2019-09-1811:43kszaboOh, in that case I would write a CLJS pathom remote for Fulcro and fire off Firebase calls in the mutation resolvers#2019-09-1811:44kszaboI thouhgt you were planning to use Pathom for the backend#2019-09-1811:46Abhinav SharmaNa, not yet - I’m inclined towards the pure frontend use cases.#2019-09-1811:47Abhinav SharmaThen these firebase calls would need to be written against the REST interface right? https://firebase.google.com/docs/projects/api/reference/rest#2019-09-1811:49kszaboprobably, this is new territory for me as well#2019-09-1811:49Abhinav SharmaMm got it, your pointers have been a good help man!#2019-09-1812:24wilkerlucio@U2BU9G0LT to help thinking about it, consider that Fulcro only communicates using EQL, so any communication has to go via it, this means keywords for requesting data and mutations to change the world (in any way, like sending messages to firebase operations), makes sense?#2019-09-1812:32Abhinav SharmaHi @U066U8JQJ, basically Pathom is the communication layer for Fulcro wrt outside world right, I get this. I guess to integrate with any rest/graphql backend it's important to teach this layer how to interpret the servers format right?#2019-09-1812:33wilkerlucioyeah, correct, I prefer to say EQL because Pathom is an impl of it, but you can make your own if you want, that is no direct thing connecting fulcro and pathom, instead both talk EQL#2019-09-1812:34wilkerlucioI think the distinction is important, because the language is formal and helps to understand the boundaries#2019-09-1812:35Abhinav SharmaCool, will remember this :) Thanks!#2019-09-1913:59Abhinav SharmaHi guys, I’ve been exploring the pathom/http/fetch api but having problem translating the clj-http code to the fetch code
(-> (http/request {:method  :get
                     :as      :json
                     :url     requestbin-url})
      :body)


pathom fetch version, I’d only like to extract the resulting :body from this but due to async nature only a channel gets stored in outer var
(go
  (<? (fetch/request-async {::http/url    requestbin-url
                            ::http/as     ::http/json
                            ::http/method "get"})))

Suggestions are welcome šŸ™‚
#2019-09-1914:12wilkerlucio@abhi18av that's the nature of async things, if you use pathom async or parallel parser you can return channels and pathom will process, async is contagious (like macros), once there you need to deal with it#2019-09-1914:21wilkerlucio#2019-09-1914:27wilkerlucioHello friends, for everyone here that asks, or try to use Pathom with Datomic, I've started working in a plugin to facilitate the integration between those two things: https://github.com/wilkerlucio/pathom-datomic#2019-09-1914:27wilkerlucioI personally don't use Pathom with Datomic myself (I'm the middle game business, calling services) so feedback is very appreciated#2019-09-1914:28wilkerluciocurrently it can auto-handles queries using :db/id or unique attributes, also helpers to query for entities automatically leveraging user sub-query, full docs on README#2019-09-1914:30wilkerlucioI hope you enjoy šŸ™‚#2019-09-1914:56kszabowe do, thanks for the great update!#2019-09-1914:58eoliphant@wilkerlucio let me know if you need any help on that, we use them together quite a bit#2019-09-1915:03eoliphantjust went thru the docs, pretty nice stuff šŸ™‚#2019-09-1915:03wilkerluciothanks, for now would be great if you can try it out, and find the gaps/problems so we can iterate on it#2019-09-1915:03eoliphantwill do#2019-09-1915:04eoliphantI wish the client API supported filter, that’s a pretty cool way to support security#2019-09-1917:36wilkerlucio@eoliphant with the plugin we can have full control, I'm still wondering about ideas to make it, the most obvious are white/black listing for some attributes, @tony.kay other day also gave the idea to have some sort of nav? fn interface, where you get the context and decide if that attribute is allowed or not in that context (so far seems the most extensible idea)#2019-09-1922:40eoliphantyeah that also would work. attributes are easier i guess, but then there’s then need to limit based on ā€˜ranges’, etc#2019-09-1922:41eoliphantlike I can see :person/name but only for entities where some other conditions are met#2019-09-2009:15kszaboany ideas to improve performance of trivial resolvers (think alias-resolver) better when using core.asyns’s threadpool for execution? My usecase is that on my entity I have a lot of entries with keys where I wrote an alias resolver for each one. Pathom is essentially doing the job of clojure.set/rename-keys at this point, but the context switches kill the performance (since there are a lot of tiny resolvers to execute)#2019-09-2009:16kszaboI could write a single resolver which does the job with clojure.set/rename-keys but since there is no support for optional inputs (there is an issue for that) it cannot be currently expressed#2019-09-2017:58wilkerlucio@thenonameguy one question first, why do you need so many renames? if you are loading from some source that doens't have them, is it possible to convert when you fetch it on the first time?#2019-09-2017:58kszaboit’s part of our domain#2019-09-2017:58kszabowe had a short uuid based name for some of our fields#2019-09-2017:58kszaboand there is a corresponding human name#2019-09-2017:59wilkerluciook, that's a valid case, just not optimized if you using a lot of them#2019-09-2017:59wilkerlucioanother thing to consider is, do you need the parallel parser? although I keep recommending it as the default, I'm starting to believe that for most people the serial parser may be a better option#2019-09-2018:00wilkerluciothe question is, does the queries you run have parallelism opportunities? usually if its a single or a few datasources that may not be the case, and them you are just paying the higher overhead and complexity price of using the parallel parser#2019-09-2309:20kszaboI think a better alternative would be to tag certain resolver as ā€˜sync’ in the resolver map. These resolvers then could be evaluated inline in the go-loop to avoid asynchronous overhead.#2019-09-2314:57wilkerlucioyeah, I was imagining that this could be realised automatically, after the first run pathom could "notice" this is sync, and its fast, using those variables it could automatically choose to not use the thread pool#2019-09-2314:57wilkerluciowhat you think?#2019-09-2315:45kszaboI think that works, also this improvement could be automatically applied to some resolver helpers like alias-resolver* (if you don’t consider this a breaking change)#2019-09-2315:45kszaboan alternative would be to use the weights of resolvers and have a cutoff where resolver-weight < weight-cutoff is executed serially#2019-09-2315:46kszabothis could be really small by default#2019-09-2317:15wilkerlucioyeah, that's what I was thinking šŸ™‚#2019-09-2317:16wilkerlucioan even bolder impl could cache "common paths", kind like an internal JIT, get long processing chains and pre-comp the fns#2019-09-2318:29kszabospecter does something like that AFAIK for it’s transforms#2019-09-2318:56wilkerlucioyeah, I guess it does, but there you have to manually build the transformation pipeline, what I'm thinking here is pathom seeing common paths (because they will naturally tend to repeat) and cache that based on usage, makes sense?#2019-09-2309:20kszaboI think a better alternative would be to tag certain resolver as ā€˜sync’ in the resolver map. These resolvers then could be evaluated inline in the go-loop to avoid asynchronous overhead.#2019-09-2114:28cjsauer@wilkerlucio I’m interested in integrating pathom with datascript in a manner similar to what you’ve done with pathom-datomic. Given acceptable code, would you accept a PR for that, or do you think that should exist in its own pathom-datascript repo?#2019-09-2116:10wilkerlucio@cjsauer that sounds cool! I think a separated repo is better in this case, even though they look similar, they are different things#2019-09-2116:11cjsauerYeah that seems reasonable. I have a prototype working right now, but it’s a bit rough. Still testing…#2019-09-2121:11cjsauerSo, it works, but it obviously requires that you specify everything in the schema. In datascript, it’s common to specify only :db/unique, :db.cardinality/many, and :db.type/ref attributes; everything else goes unspecified. This means that using the plugin requires ā€œpaddingā€ the schema with the attributes you’re expecting to resolve, e.g.
(def schema {:todo/title {}})
Seems one way to automate this is to perform resolution (i.e. (ds/pull ...)) in the ::pc/compute-output handler in order to determine what exactly datascript already knows. Then ā€œsubtractā€ the parent query from that. This is my first look at pathom internals, but for some reason performing resolution in order to compute outputs feels…weird :thinking_face:
#2019-09-2121:12cjsauerRepo is here: https://github.com/cjsauer/pathom-datascript#2019-09-2317:58cjsauer>This is my first look at pathom internals, but for some reason performing resolution in order to compute outputs feels…weird Just had a thought, would datascript integration be better implemented as a custom reader? I can’t help but feel that this approach is similar to the built-in map-reader. Datascript appears to omit values it doesn’t have when using pull. So then I imagine a custom reader could be developed that simply performs the pull on the root/parent query, and then just hands that map to the normal map-reader…and resolution would continue normally from there.#2019-09-2318:01wilkerlucio@cjsauer I guess that can work, a concern I can think is that Pathom does a lot of merging/changing the map as it progress, there is a risk at some point it would try to merge and get a regular map out, and then you can't keep pulling from datascript from that#2019-09-2318:02cjsauerI’m imagining the datascript pull happening once, only on the root query. Datascript’s pull is roughly a resolver parser on its own, meaning it can digest eql queries out-of-the-box. So I suppose maybe a reader isn’t the correct place for this, because they are called on every step of resolution, correct?#2019-09-2318:03wilkerlucioah yeah, thats how people have been doing datomic so far, and should work well with datascript too#2019-09-2318:03wilkerlucioand helps with auto-complete (as you declare the resolver outputs)#2019-09-2318:06cjsauerPart of the challenge here, and where my understanding of Pathom is lacking, is that you can’t know the outputs until you’ve already performed the pull. With Datomic, the schema is all-knowing, but with datascript, the schema is completely optional.#2019-09-2318:10cjsauerIs there a way to do this without declaring the outputs? I think I remember reading that you can include data in the env that you already have in-hand. Maybe I could perform the pull, assoc that result into the env/context, and then invoke the parser. The map-reader would then pull what exists in the env, and will use resolvers for everything else.#2019-09-2318:13cjsauerI’m unsure tho whether pathom lets you specify ::pc/input of some attributes without specifying those attributes in some respective ::pc/output…#2019-09-2318:36wilkerlucio@cjsauer yeah, its a bit of tricky thing, if we don't know anything ahead of time we cant predict the paths#2019-09-2318:46wilkerluciowell, can work, just not get proper auto-complete#2019-09-2318:46cjsauerFrom your datomic plugin, I saw that ::pc/compute-output is a possible hook for plugins, so I suppose a datascript plugin could dynamically compute the output from the pull result. Is there an existing function that takes a map and returns its respective eql query? The ::pc/compute-output hook for datascript would then be something like (-> (pull db parent-query ident) map->query)) where an example of map->query would be:
(map->query {:a 1 :b 2 :c {:d 3}})
[:a :b {:c [:d]}]
Is this the return value that ::pc/compute-output is expecting?
#2019-09-2318:49wilkerlucio@cjsauer there is pc/data->shape that does what your map->query there is doing, a issue I see is that then you would have to compute it multiple times, depending on the query size that may be unwanted (and consider that compute-output may be called multiple times in some edge cases), also, compute-output has nothing to do with auto-complete, that is still a separated thing, that's provided using the ::pc/index-io#2019-09-2318:50cjsauerI see. So the index would still be incomplete with this solution.#2019-09-2318:51cjsauerI also don’t like it because it means that output computation depends on the db…which feels wrong.#2019-09-2318:51wilkerlucioyeah, auto-complete currently doesn't have support for a dynamic schema like datascript, the keys have to be known ahead of time#2019-09-2318:51wilkerlucioI still have to complete the datomic impl on that, auto-complete is not there (filling the index-io, but that's trivial to implement in the datomic case)#2019-09-2318:52wilkerluciomaybe would be a good idea to have the schema for the datascript anyway? can be as simple as listing the keys you wanna use there#2019-09-2318:53cjsauerYeah that seems to be the direction this is leaning. Having a schema is probably good practice anyway…plus with a datomic backend the respective datascript schema is trivial to derive.#2019-09-2318:53cjsauerWould just be super cool to automate the parts of app state that are client-only#2019-09-2319:01wilkerlucioyup, and just out of curiosity, so you are using datomic on the server, and also having a datascript partial copy on the client?#2019-09-2319:08cjsauerYeah, and with these two pathom plugins I could avoid writing a substantial number of resolvers. Instead, I’d just need to specify my attributes in a central schema, and would get all of this query power ā€œfor freeā€, with the ability to query for out-of-band/derived data using hand-written resolvers. The reason for datascript on the client is drastically simpler client-side merges of remote data (literally just a transact! call), and also the ability to use datalog in resolvers is pretty sweet. Not to mention unified server/client paradigms.#2019-09-2319:11cjsauerI used to develop apps with Meteor, and ever since then the idea of using a matching client-side database (in Meteor’s case it was Mongo + MiniMongo) has been very attractive to me. It allows the framework to do additional heavy-lifting in certain cases.#2019-09-2319:14wilkerluciointeresting, do you already have some feelings about how Fulcro + Pathom approach compares with Meteor? specially interested in cool features that you had on Meteor and miss with Fulcro + Pathom#2019-09-2319:20cjsauerOne of the coolest features that Meteor had was the ability to have real-time data sync working in about 30 seconds. They would tail the Mongo op-log, and sync database changes over a ws in real-time. Because the client was also using (Mini)Mongo, reconciling those changes was really simple. I haven’t used Fulcro enough to know what kind of real-time sync capability it has out-of-the-box, but that was surely a super power in Meteor. With the default app you could make a change in one browser, and see it reflected in another. The analog with this setup would be taking the :datoms from datomic’s tx-result map, and then syncing those down a ws to datascript. Obviously you’d need to be careful with syncing private datoms down the wire, but I think these security concerns could be encoded in the schema.#2019-09-2319:28wilkerluciothe hard part on real time is more in the realm of consistency, in theory you could just do a query and "watch" it, the problem comes as: how do I know that query updated?#2019-09-2409:12kszaboI think reactive dataflows solve this exact problem: https://github.com/sixthnormal/clj-3df#2019-09-2319:28wilkerlucioif you have a way to answer that, pulling the data and populating the UI is trivial, but that question is hard to answer, hehe#2019-09-2320:04cjsauerYeah definitely. I think the scope of that problem can at least be limited. Rather than watching general queries (hard problem), you could instead just watch specific entities (which is just an ident). The latter solution, while less general, is much much easier to implement. I think Meteor also made this simplification, but they may have indeed added some kind of query-watching feature. Being able to subscribe to individual entities, while maybe not totally optimal, seems like it would solve 90% of cases when you really just want entity syncing.#2019-09-2321:08wilkerlucioyeah, if I were to port that idea, I would subscribe to idents, like [:my-company.product/id 123]#2019-09-2409:12kszaboI think reactive dataflows solve this exact problem: https://github.com/sixthnormal/clj-3df#2019-09-2413:25cjsauerI looked into this, and even attempted to read the paper after it was presented at Conj 2018. It’s very cool, but something about it feels like using a sledgehammer to kill a fly. It’s likely that I don’t truly understand how it works, coupled with the fact that it’s implemented in a different language (Rust). That’s a fair bit of complexity to bring in to a ā€œbasicā€ application in my opinion.#2019-09-2416:45cjsauerWhat is the intended result for an entity that is not found? For example, when I do (parser {} [{[:my/id 10] [:my/id]}]), I always end up with {[:my/id 10] {:my/id 10}}, which I’m sure is ā€œcorrectā€, but how do I express the fact that [:my/id 10] does not exist? I’d expect something like {[:my/id 10] ::p/not-found}.#2019-09-2416:51souenzzonot-found can mean "1- can't find a resolver to this keyword" or "2- the resolver didn't return that key" on both cases, pathom will solve it for you, and you can attach a "elide-special-keys" plugin to remove that keys#2019-09-2416:57cjsauerAh, yep, adding p/elide-special-outputs-plugin to my plugin list did the trick. Thank you.#2019-09-2422:46wilkerlucioone thing that is good to clarify around "presence check" is that Pathom works around the idea of attributes in the first place, so the real check must be per attribute (and not per entity), this is an important distiction, for example, you can have multiple endpoints hooked around some id like :customer/id, in some resolvers the result might be unknown and others may work, so I think its important to keep in mind this "attribute individuality" mindset#2019-09-2512:49cjsauerThat’s a really good point. Do you tend to use the elision plugin @wilkerlucio ? It seems the trade-off is more defensive/conditional code on the client, rather than nil punning on the results (not using the plugin versus using it). #2019-09-2512:51wilkerlucio@U6GFE9HS7 yes, using elision is fine, I do most of the time too, I like to keep the default without it so people understand if you need this level of detail you can get it, but to be honest I'm still trying to find cases that I prefer not eliding, that default recommendation may change šŸ™‚#2019-09-2421:06BrianHow do I call the person-name-resolver found here: https://wilkerlucio.github.io/pathom/#_derivedcomputed_attributes in the second code chunk? I've tried as many ways as I can think of and can't seem to get it. My most recent attempt was (<!! (parser {} [{[:person/first-name "Brian" :person/last-name "Last"] [:person/full-name]}]))#2019-09-2421:06BrianHowever that just returns a not-found#2019-09-2421:23souenzzo(<!! (parser {:person/first-name "Brian" :person/last-name "Last"} [:person/full-name])) @brian.rogers#2019-09-2421:30Brian@souenzzo that yields a "not-found" still for me#2019-09-2421:30BrianI copied and pasted that. Are we trying to put that directly into the context map with your code there?#2019-09-2421:32souenzzoyep#2019-09-2422:05BrianHow can I do it as part of the query instead of the context?#2019-09-2422:05BrianIs that possible?#2019-09-2422:46wilkerlucioone thing that is good to clarify around "presence check" is that Pathom works around the idea of attributes in the first place, so the real check must be per attribute (and not per entity), this is an important distiction, for example, you can have multiple endpoints hooked around some id like :customer/id, in some resolvers the result might be unknown and others may work, so I think its important to keep in mind this "attribute individuality" mindset#2019-09-2422:48wilkerlucio@brian.rogers I think you are looking for this: https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/sugar.html#_parser_context_interface#2019-09-2422:49wilkerlucioyou can also do it at query level using an ident query, that's described here: https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/resolvers.html#_multiple_inputs#2019-09-2422:49wilkerluciois this what you are looking for?#2019-09-2512:49cjsauerThat’s a really good point. Do you tend to use the elision plugin @wilkerlucio ? It seems the trade-off is more defensive/conditional code on the client, rather than nil punning on the results (not using the plugin versus using it). #2019-09-2515:34Brian@wilkerlucio I might be wrong but I have this idea where I'll be able to take my users' query right off the wire and pipe it into pathom using '(<!! (parser {} query))` and that worked fine but now I'm trying to build a query which allows me to input a first and last name so that I can call :person/full-name. It sounds like multiple inputs is the answer but I don't have an ident to work with like the :customer/id equivalent. The real problem I'm trying to solve is I want my users to be able to give me an IP and a port and I can tell them if that combination is in our database.#2019-09-2515:38BrianI can do this with (<!! (parser {:pathom/context {:entry/port "433" :entry/ip "127.0.0.1"}} [:entry/blacklist?])) BUT that extends beyond them sending a query because I'd also have to parse out their context and I want my users to only have to send me the query. Is that possible? If so, what is the query I would use to call the person-name-resolver here https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/resolvers.html#_derivedcomputed_attributes?#2019-09-2516:36wilkerlucio@brian.rogers in the second link I sent to you, if you are using connect, you can do queries like this:#2019-09-2516:36wilkerlucio
(parser {} '[{([:person/id 123] {:pathom/context {:person/first-name "Brian" :person/last-name "Last"}})
              [...]}])
#2019-09-2516:36wilkerlucionote that we still need an ident, its important to separate the "identity" from the record, if there is no identity at all, then anything that you put there should work anyway#2019-09-2519:27BrianThanks @wilkerlucio! Pathom is truly incredible#2019-09-2715:25msswhat’s the best way to handle returning idents to fulcro out of my pathom resolvers? adding a [:thing/by-id thing-id to a return value throws an error, ostensibly because pathom is further trying to parse that ident? would appreciate any direction or input#2019-09-2715:26mssobviously if thing is a deeply nested structure, I don’t want to have to pull all of that and re-parse it in my resolver and let fulcro normalize it on the client. would prefer to just spit back an ident that can be incorporated into my fulcro db as is#2019-09-2716:00mssmy precise issue is trying to return something from my thing resolver that looks like {:thing/property-a 123 :thing/creator [:users/by-id #uuid ....]}#2019-09-2719:02wilkerlucio@mss I don't get it yet, can you please show some example query and data of what's happening?#2019-09-2720:08msssure, hope this makes sense: so I have a query generated from a few query components that looks roughly like the following:
{:project/test-runs [:test-run/id
                     :test-run/progress
                     :test-run/status
                     {:test-run/test 1}
                     {:test-run/initiator 1}]}
the :test-run/test and :test-run/initiator are set to 1 bc as I understand it, this prevents circular references to things that reside elsewhere in my db and are served by other queries the result spit back from my resolver is something like:
{:project/test-runs
 [{:test-run/id #uuid "537d4901-7bcc-4201-b5e8-71670d0b4309",
   :test-run/progress 0,
   :test-run/status :test-run-statuses/not-started,
   :test-run/test [:tests/by-id #uuid "5b16d210-f826-4823-9573-83240c8e41fa"],
   :test-run/initiator [:users/by-id #uuid "c926d210-f826-4823-9573-83240c8e28s2"]}]}
receiving a reader error within pathom, ostensibly for those two properties that java.lang.IllegalArgumentException: Don't know how to create ISeq from: java.lang.Long
#2019-09-2720:08mssclearly I’ve misunderstood something about circular refs in queries and how those would translate to pathom#2019-09-2720:22noonianI think the problem is that {:test-run/test 1} and {:test-run/initiator 1} is not valid syntax in EQL. I'm not sure about the circular references thing you are refering to, but maps are used to represent joins (and unions) so pathom is probably expecting 1 to be a vector of attributes to read from the test and initiator and that is what is causing the error#2019-09-2720:29wilkerluciothe query is valid, its a recursion with limit, the problem is with your returns, the values should be maps instead of idents, try making your data looks like this:
{:project/test-runs
 [{:test-run/id #uuid "537d4901-7bcc-4201-b5e8-71670d0b4309",
   :test-run/progress 0,
   :test-run/status :test-run-statuses/not-started,
   :test-run/test {:tests/id #uuid "5b16d210-f826-4823-9573-83240c8e41fa"}
   :test-run/initiator {:users/id #uuid "c926d210-f826-4823-9573-83240c8e28s2"}}]}
#2019-09-2720:47mssmakes sense. so that works as far as satisfying pathom, unfortunately it looks like my mental model of how to avoid circular refs in fulcro queries is a little busted. going to pop on over to that channel and dig in#2019-09-2720:47mssthanks for the help y’all#2019-09-2721:06wilkerlucioimagine that for every join you are changing the context, when we do use idents like [:person/id 123], what we are effectively doing is start a context with {:person/id 123}, so when you return maps to get further processed, they are the initial context for the join and pathom will use the resolver data to figure it out, makes sense?#2019-09-2921:48cjsaueršŸ‘‹ Wilker. Is there a function in pathom that can ā€œshrinkā€ a query/AST, meaning determine redundancy, shared paths, etc? Trivial (naive) example:
[:user/name :user/name :user/name]
;; becomes
[:user/name]
#2019-09-3003:32wilkerlucionothing built in#2019-09-3013:30pvillegas12How can disable caching behavior for the pathom resolver in development?#2019-09-3014:40wilkerlucioyou can set ::pc/cache? false on the resolver configuration, but usually you should not need to disable it, the cache is only per-request, special cases are resolvers that depend on something from the environment#2019-09-3014:40wilkerlucio(and that env thing is changing between calls to this same resolver)#2019-10-0309:18moocarHey folks, I have a pathom parser that uses a shopify index, and adds additional resolvers. I want to reference the parser inside the resolvers so I can make the request to shopify. Is this possible? I'm pulling the parser out of the env like below, but I'm getting an unhelpful NullPointerException deep in the parser
(pc/defresolver resolver [env _]
  {::pc/output [{::foo/all [::foo/id]}]}
  (go {::foo/all (<! ((:parser env) {} [{(:shopify/..) [...]}]))}))
#2019-10-0309:19moocarThe only way I can get it to work is by constructing a separate parser for the shopify index and passing that into the top level parser ::p/env#2019-10-0313:05souenzzo((:parser env) {} should be ((:parser env) env [{(:shopify/... )}] part also unquoted. so it should be resulting in a [{nil ...}] query#2019-10-0320:59moocarThanks!! That did the trick, at least for queries by ident. For some reason global queries (quoted) [{(:shopify/customers {}) [...]}] are returning {} every time, even when the exact same query works from fulcro inspect query. It might be something unrelated though.#2019-10-0319:21Abhinav SharmaHello guys, The PartiQL query language from AWS seems like a good target for pathom adapters - would love to hear your thoughts on this https://partiql.org/#2019-10-1709:37kszaboHey @wilkerlucio. I’m proposing to my company that we should use EQL/Pathom for aggregating our microservice graphs/general RPC mechanism. Do you have any comments on this approach/any pitfalls?#2019-10-1709:43kszaboThe other alternative would be GraphQL via Lacinia#2019-10-1709:43kszabobut I really prefer the approach taken by EQL+Pathom#2019-10-1713:57Mark AddlemanWould you summarize your thoughts on why you prefer EQL + Pathom?#2019-10-1714:07kszaboIt’s mainly it’s attribute focus compared to entity-orientation. Also that the query language is data is a big plus, since you can trivially concat/merge/dissoc queries when you are building them up. The lack of type system is also a plus as I don’t believe that type systems capture enough meaning that they are useful outside of the most trivial cases (oh I expected an int for :user/age and got a float or whatever), the web isn’t built with typed attributes but with dynamic adapting systems.#2019-10-1714:18Mark AddlemanThanks#2019-10-1714:03myguidingstar@thenonameguy if Clojure is not your company's main language, then it's hard to adopt EQL as the interface language. But even in that case, you can disguise as GraphQL :)) https://github.com/denisidoro/graffiti#2019-10-1714:04kszabowe are 100% clojure#2019-10-1714:09myguidingstarthen I find no reasons for not using EQL#2019-10-1714:09kszabonice library though, thanks for putting it on my radar! I had similar iffy feelings towards lacinia when reading it’s readme#2019-10-1714:10kszabothe ecosystem support for GraphQL seems like a safer choice, but I woulnd’t mind developing tools as I got time from my employer working on things like this#2019-10-1714:15wilkerlucio@thenonameguy considering you are all Clojure, I think Pathom is a good one, IME people tend to get quickly how to write resolvers. one common pitfall I see people doing sometimes is trying to make a resolver do too much, in those cases is good to instruct the use of smaller resolvers, adding more names to the system, as each name turns into an entry point for re-use, having more smaller resolvers is preferable#2019-10-1714:16kszaboyup, I got this embraced by most devs, we are already reimplemented 2 microservice’s all REST endpoints of ours with Pathom parsers + exposing Pathom#2019-10-1714:18wilkerluciocool, just to clarify, are you using pathom on each service, and making endpoints by calling the parser, or just regular rest services and a separated service with pathom connecting them?#2019-10-1714:19kszaboboth#2019-10-1714:19kszabothe first for backward compatibility#2019-10-1714:19kszabobut we want to do the latter#2019-10-1714:20kszabohave a set of microservices which are connected by Pathom in the customer-facing service#2019-10-1714:20kszabowith a merged graph#2019-10-1714:21wilkerlucioyeah, I ask because I have though a couple times about how would work to do Pathom <-> Pathom integrations, I mean, is theoretically possible to make one pathom parser pulls the index of another pathom parser, and "auto-connect" them, I just didn't had the opportunity to try that out, but if that's something you are looking for we could progress on that idea#2019-10-1714:21wilkerluciosimilar to how Pathom integrates with GraphQL today#2019-10-1714:21kszaboyes, I saw some tickets on Github#2019-10-1714:22kszaboI would be happy to contribute#2019-10-1714:22kszaboalso I’m working on a tool where you can use the Index/Graph explorer without multiple remotes#2019-10-1714:23kszaboand if it’s possible merge the supporting datastructures to create a hollistic view#2019-10-1714:23wilkerluciofor the index explorer that can be easy#2019-10-1714:23kszaboyeah that would be the first step#2019-10-1714:23wilkerluciothere is pc/merge-indexes you can use#2019-10-1714:23kszaboto let stakeholders have greater visibility what is available#2019-10-1714:25kszabomy final goal would be that our services communicate with strong keyword names, and the remote/local part of the computations is configuration dependent#2019-10-1714:25kszabomaking a monolithic/microservices architecture possible#2019-10-1714:29wilkerlucioexciting stuff šŸ™‚#2019-10-1714:30kszaboit is, but if IIRC you posted a plugin where the parser chose a different server based on something. So I assumed you are doing something similar#2019-10-1714:31kszabobut I guess you are just doing microservices/DB -> REST/DB -> frontendserver -> Pathom -> UI#2019-10-1714:34wilkerlucioin my case I'm doing just HTTP calls to other services#2019-10-1714:36kszaboHTTP+(not pathom)#2019-10-1714:36kszaboI want to do HTTP + Pathom#2019-10-1714:36kszabowith the possibility of removing the HTTP part if the remote query can be fulfilled locally#2019-10-1714:41kszaboAlso one pattern that will be important in our queries is a sense of ordering. Ie. I want my: Query:
[{[:user/id 42] [{:non-existing-pathom-feature/ordered-shortcircuit [:cache/user-name :expensive/user-name]}]}]
Response A:
[{[:user/id 42] {:expensive/user-name "Expensive Name"}]}]
Response B:
[{[:user/id 42] {:cache/user-name "Cheap Name"}]}]
#2019-10-1714:42kszaboIs this something that could be included as a Pathom utility instead of rolling our own impl? (Once we do this)#2019-10-1714:42kszaboof course we could implement all of the combinations with separate resolver, but that blows up the resolver count with query resolver details#2019-10-1714:43kszabowe are doing the latter currently btw, so we have a :cache->expensive/user-name resolver#2019-10-1714:46wilkerluciohumm, for caching I think would make sense to just overload the name, because this is a implementation detail, the user should not care if that comes from cache or not (unless there is a business value on switching between them)#2019-10-1714:46kszabothere is#2019-10-1714:46kszaboto save COGS#2019-10-1714:47wilkerlucioCOGS?#2019-10-1714:47kszabohttps://www.investopedia.com/terms/c/cogs.asp#2019-10-1714:47kszaboto save šŸ’ø#2019-10-1714:48wilkerlucioabout ordering, this is a current problem, because you can't guarantee that the cache one will run before, but the good news is that I'm currently working to make something like that possible, I want to be able to say: always run resolver X before resolver Y (in terms of priority, given they respond for the same attr)#2019-10-1714:48kszabothat would be sufficient I think#2019-10-1714:49kszabowould this be a static thing?#2019-10-1714:49kszabo[:fast-but-inaccurate :moderate-but-more-accurate :slow-but-precise]#2019-10-1714:50kszabogiven these resolvers different contexts might want the middle first#2019-10-1714:53kszaboit’s fine if you can define a default order at parser creation time, but there should be an escape hatch for cases like this#2019-10-1714:53kszabootherwise plain declarative queries won’t suffice for more advanced use-cases#2019-10-1716:14wilkerlucio@thenonameguy yes, I wanna make it a static thing, but I was thinking something that's not so global, there is no "one priority list", but instead would be relative definitions, not sure about the final interface yet, but something like {'some/resolver {::pc/run-before 'some/other-resolver}}#2019-10-1812:49souenzzoHey I'm developing a application dashboard, and trying to keep api-url as a "first class" My query start with [{[:app/api-url "..."] [{:app/things [:thing/id :thing/stuff]}]}] My first resolver do ::pc/input #{:app/api-url} ::pc/output [{:app/things [:thing/id]}] Then it should call ::pc/input #{:app/api-url :thing/id} ::pc/output [:thing/stuff] But there is no api-url in {:thing/id ..} input There is some helper to "foward" some attributes?#2019-10-1814:18wilkerlucio@souenzzo this looks more like an environmet data, do you see :app/api-url is an actual entity in your system, or this just data you need to flow down?#2019-10-1814:21souenzzoI'm not sure šŸ˜… I started using it as a environment, associated with the browser session, but then I think "that if I want show a list of things where there is things from 2 instances?"#2019-10-1817:15souenzzoIm trying to implement a plugin/reader that keep some keys in the new contexts#2019-10-1818:04souenzzodoes it looks OK? https://gist.github.com/souenzzo/b394ee85932d8246b0a465a42635f9a5#2019-10-1818:16wilkerlucioits ok, but still feels like this should be something you set on the environment, it naturally flows down, and you can change it in the middle of queries using ::p/env on the return of some data if you need to#2019-10-1818:16wilkerluciowe do something similar like that at nubank to deal with shards#2019-10-1819:30souenzzothis p/env should exists by defaul?#2019-10-1819:32souenzzohttps://wilkerlucio.github.io/pathom/#_example_shard_switch#2019-10-1820:06wilkerluciothe ::p/env thing may be missing docs (sorry), but in a gist, you do something like this in your resolver return value: {:some/data 123 :deep/info {:more/data "meh" ::p/env (assoc env :app/api-url "new-url")}}#2019-10-1820:06wilkerlucionow when it gets to process :deep/info, env will be changed, makes sense?#2019-10-1823:12cjsauerCan a parser reflect on itself? As in can I query for things like resolvers, output keys, etc?#2019-10-1823:13cjsauerI imagine the viz uses something like this, but I can’t quite grasp the gist of it#2019-10-1905:15Spencer AppleHey! Thanks for your work on Pathom! I am struggling with two things and if you have some time I would love some help. 1. what's the best way to pass a request env (like ring's) into pathom?
;; I call this api-parser via the fulcro websockets networking
(defn api-parser
  ([query] (pathom-parser {} query))
  ([env query]
   ;; should I just pass the env directly here?
   (pathom-parser {:user (:user (:request env))} query)))
2. I have the following resolvers and am making a query which to me makes sense, but it's not returning my room.
(pc/defresolver room-resolver [env input]
  {::pc/input #{:room/id :user/id}
   ::pc/output [:room/id
                :room/status]}
  (util/log "calling room-resolver")
  (get-in @state/room-table [(:user/id input) (:room/id input)]))

(pc/defresolver user-resolver [env input]
  {::pc/output [:user/id]}
  {:user/id :base-state})
Then I can run the query which finds the user-resolver but not the room resolver
(api-parser [:user/id {[:room/id :] [:room/id :room/status]}])
;; => {:user/id :base-state, [:room/id :] :com.wsscode.pathom.core/not-found}
My understanding is that pathom should realize that the room resolver can be fulfilled since there is a :user/id and :room/id. Any ideas?
#2019-10-1912:30wilkerlucio@cjsauer yes, you can query for the indexes, that's how the auto-complete for queries currently works, it downloads the indexes and show it, there is also the index explorer: https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/exploration.html#2019-10-1912:32wilkerlucio@splayemu to send stuff to env, use the first argument of the parser, as you used, I suggest you pass the whole request, like: (pathom-parser {::request request} [...])#2019-10-1912:32wilkerlucioalso, remember to use qualified keywords to avoid collisions#2019-10-1912:32wilkerlucioabout the resolvers, they look strange as is#2019-10-1912:33wilkerlucio::pc/input #{:room/id :user/id} is not a good way, you are making a dependency hard to reach, if you can use a single id would be great (and have normalized structures to request from)#2019-10-1917:18Spencer AppleThanks, yeah as you suggested, it does make sense to use a single id. I turned it into a single id and built a resolver to combine the user-id and room-id:
(pc/defresolver user-room-resolver [env {:keys [room/id]}]
  {::pc/input #{:room/id}
   ::pc/output [:user-room/id]}
  (let [user-id (-> env :user :user/id)]
    {:user-room/id [user-id id]}))
#2019-10-2001:55wilkerlucio@splayemu this still looks strange, what is :user-room/id? did you mean to get the id from the room? if that so, your output should be like: {:room/user {:user/id id}}, makes sense?#2019-10-2014:03Spencer AppleNo since I queried into my room table first by :user/id and then by :room/id I concatted the ids into an ident. [:user/id :room/id]. For now I am just using atoms to store my data so the query is just (get-in @room-table [user-id room-id)#2019-10-2014:04Spencer AppleThank you for all the help understanding pathom!#2019-10-2009:24schmeeHi! is it possible to programmatically generate resolvers? I want to try to generate resolvers from a SQL database using foreign keys to resolve between tables#2019-10-2009:51schmeehmm, judging by what defresolver actually creates it looks like I can just create a #:com.wsscode.pathom.connect{:input ... :output ... :resolve ...} map myself, right? šŸ™‚#2019-10-2009:53schmeeahh, just found https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/shared-resolvers.html#2019-10-2010:02schmeeA question about query optimization: I have to two tables, user and organization, and each user has an organization_id that links between the two. I’ve set up some basic resolvers and it works great, but it runs two queries: - SELECT ... FROM user WHERE id = ? to get the user data and organization id - SELECT ... FROM organization WHERE id = ? to get the organization data Is it possible to somehow combine these into
SELECT ... 
FROM user u 
WHERE u.id = ?
INNER JOIN organization o ON u.organization_id = o.id`
I have tables that are linked like this ā€œN stepsā€, and it would be fantastic if I could combine it into one query with ā€œNā€ joins instead of ā€œNā€ separate queries.
#2019-10-2010:03schmeeI looked at batch resolvers, am I correct that they are more for avoiding multiple calls when you have multiple results from your query, rather than avoiding joins?#2019-10-2010:20schmeeand btw, can I just say that Pathom is absolutely amazing 🤯#2019-10-2012:09schmeeOne more question: is it possible in a resolver to know which outputs are requested? For example, I have a resolver that provides all the data in a table using SELECT * FROM .... But often you only need one or two outputs from a resolver to fulfill the query, and it would be more efficient to do SELECT column1, column2 FROM ... instead. But I couldn’t find anything in the env that say ā€œthis is the data I need from the outputs to fulfill the this specific queryā€, is there such a thing?#2019-10-2106:17myguidingstarare you looking for :com.wsscode.pathom.core/parent-query in resolver's input env?#2019-10-2012:14schmeeand sorry for the wall of questions šŸ˜…#2019-10-2017:28adamfeldman@schmee you may be interested in Walkable, which uses Pathom to fetch data from SQL databases: https://github.com/walkable-server/walkable#2019-10-2017:31schmeeI gave it a run earlier, but it lacks some things that I need: - no resolver support, so you have to explicitly specify your ā€œNā€ joins in your queries (dealbreaker for me) - no optimization like the one I described above: it does one query per table#2019-10-2017:32adamfeldmanThank you for sharing your experience with Walkable! I haven’t used it yet, so this is helpful#2019-10-2109:48henrikI’m struggling with hooking up a GraphQL API (on the backend), and I could really use some help at this point. The GraphQL API acts as an interface to an ElasticSearch cluster. It’s purely read-only, no mutations. To start at the end, I’d like to pass this as an MVP and receive a response:
[{(:zd/random_articles
   {:sample_size 3})
  [:article_doi]}]
This is probably the simplest endpoint in our API. It takes an int and returns the equivalent number of random articles. This is the setup:
(defonce indexes (atom {}))

(def zd-gql
  {::p.http/driver p.http.clj-http/request-async
   ::p.http/headers {"x-auth-token" (auth-token)}
   ::pcg/url api-url
   ::pcg/prefix "zd"
   ::pcg/ident-map <something, haven't worked it out yet>})


(defn load-graphql-index! []
  (pcg/load-index zd-gql indexes))
(I call load-graphql-index! at server startup, and discovered that it has to be called after the parser is initialized, or it doesn’t work.) This is the parser:
(def pathom-parser
  (p/parallel-parser
	{::p/env     {::p/reader [p/map-reader
							  pc/parallel-reader
							  pc/open-ident-reader
							  p/env-placeholder-reader]
				  ::p/placeholder-prefixes #{">"}
				  ::p.http/driver          p.http.clj-http/request-async}
	 ::p/mutate  pc/mutate-async
	 ::p/plugins [(pc/connect-plugin {; we can specify the index for the connect plugin to use
									  ; instead of creating a new one internally
									  ::pc/indexes  indexes
									  ::pc/register resolvers})
				  p/error-handler-plugin
				  p/request-cache-plugin
				  p/trace-plugin]}))
Almost copied wholesale from the documentation, except that I register some additional resolvers that for now are dummy-esque, but will interface with some database later on to get/set user details and similar. On the frontend, we have Fulcro. This is the entry point:
(defn api-parser [query]
  (<!! (pathom-parser {} query)))
(Since the server already makes the entire thing async, all async stuff above is obviously redundant, as evidenced by the <!! in api-parser, but I’m trying to stay close to the example to avoid confusing myself) 1. Errors When I run query->graphql on,
[{(:zd/random_articles
   {:sample_size 3})
  [:article_doi]}]
It looks right, spitting out,
query {
  random_articles(sample_size: 3) {
	article_doi
  }
}
Which I’ve passed directly to the GraphQL API to verify that it should be returning results. It does. When running it in the Fulcro Inspector (and thus through Connect etc.), I get,
{:zd/random_articles :com.wsscode.pathom.core/reader-error,
 :com.wsscode.pathom.core/errors
 {[:zd/random_articles]
  {:message "Failed to parse GraphQL query.",
   :extensions
   {:errors
	[{:locations [{:line 3, :column nil}],
	  :message
	  "mismatched input '}' expecting {'query', 'mutation', 'subscription', '...', NameId}"}]}}}}
What could be the reason for this? I looks like the GraphQL API is receiving something odd, but could it be something else? Do I need to write a resolver manually or somesuch? Can I log the outgoing calls to the GraphQL API (`:com.wsscode.pathom/trace` didn’t help me produce insight as to what is happening) 2. Autocomplete I’ve noticed that Fulcro Inspector doesn’t expose the GraphQL queries for autocompletion, but rather just [:com.wsscode.pathom.connect/indexes :com.wsscode.pathom.connect/resolver-weights :com.wsscode.pathom.connect/resolver-weights-sorted :com.wsscode.pathom/trace]. Is this down to me creating resolvers for the GraphQL queries, or should it be able to autocomplete from the knowledge of the GraphQL schema? If I need to describe resolvers for each query to the GraphQL API, what should they return? The (pc/defresolver repositories …) in the example looks like it’s there to purely autocomplete some examples? Correct me if I’m wrong. Terribly sorry for the humongous post.
#2019-10-2112:29henrikScratch both questions 1 and 2.
(defonce indexes (atom (<!! (pcg/load-index zd-gql))))
seems to have taken care of both the problem of loading the GraphQL index in order, and the autocomplete. Upon having autocomplete, I realized that the correct way to express the query is:
[{(:zd/random_articles
   {:sample_size 3})
  [:zd.Article/article_doi]}]
#2019-10-2112:30henrikThis returns the articles as expected.#2019-10-2112:33henrikAnd wow, it feels absolutely great to not fiddle around with prebaked .gql files on the backend.#2019-10-2113:48wilkerluciohello @henrik, it seems you figured that out, did you got to also understand how to setup the ::pcg/ident-map?#2019-10-2113:53henrikAt the moment I’m just marveling in what I’ve got and exploring the composability šŸ™‚#2019-10-2113:56henrikBut I do need to get down to brass tacks on that part as well.#2019-10-2114:00henrikAm I right in assuming that ident-map should be used for ā€œident-likeā€ stuff, like the ISBN of a book for example? You wouldn’t use it for sample_size in my small example?#2019-10-2114:02henriksample_size is certainly a parameter, but it doesn’t refer to anything that is unique about the result.#2019-10-2115:56henrikAlright, I think I understand the intention behind ident-map now. In our API, most functions are variable arity. I.e., they don’t take the equivalent of [:some/id 1], they take the equivalent of [:some/id [1]] (to fetch an arbitrary number of entities in one go). Is there a way to adapt for this?#2019-10-2117:18wilkerluciofor that I think just using the regular entry points as you doing is the way to go#2019-10-2117:18wilkerluciothe ident is really for identifying things#2019-10-2117:19wilkerlucioyou probably will need it later, if you want your UI to do local data updates, those are usually ground on a single entity, then the ident lookup syntax will come handy to use with Fulcro#2019-10-2114:29henrikIs there a good way to deal with repeating patterns in EQL queries? I have an argument that can grow rather large, and tends to be repeated two or three times in a query in the worst case scenario. When talking directly to the GraphQL endpoint, I would declare it a variable and send it once, however many times it would be used in the query. I don’t know if this can be expressed in EQL.#2019-10-2114:35kszaboUse functions to generate the corresponding EQL data#2019-10-2115:01henrikThe point is to decrease the size on the wire, generating the query itself is straightforward.#2019-10-2115:02henrikI.e., I’d have to make use of GraphQL’s support for variables.#2019-10-2115:03henrikhttps://graphql.org/learn/queries/#variables#2019-10-2115:06henrikGraphQL implemented it to get around the fact that the system is kind of stupid, but as a side effect, it can be used to shrink the size of calls on the wire in the case that the variable holds data that is repeated two or more times in the query.#2019-10-2115:31kszaboOh I see, yeah, EQL doesn’t have facilities for this now#2019-10-2115:43henrikGotcha, thanks!#2019-10-2117:17wilkerluciobut would not be something hard to implement, we are getting around with some pretty big queries, it helps that transit can compress the keywords šŸ™‚#2019-10-2119:24henrikYeah, that helps (in my case) frontend -> backend, but not backend -> API. It’s not as big of a problem since both backend and API are sitting in the same AWS region and AZ. But you know, it adds up when there are enough users. šŸ™ƒ#2019-10-2119:25wilkerluciohumm, so you mean to call graphql views using the eql syntax?#2019-10-2119:26henrikWell, I’m experimenting with it. #2019-10-2119:28henrikSo far, everything has been custom (classic REST) between frontend -> backend, with GraphQL queries hardcoded in .gql files. That’s not great, and I can’t call the API directly from the frontend for reasons.#2019-10-2120:37henrikIs this setup unusual?#2019-10-2209:50henrikI sat down and watched your talk, Scaling Full-Stack Applications, and realize I totally do not want to do this. šŸ˜…#2019-10-2209:51henrikNow I just want to send the query for a component and have Pathom figure out how to retrieve that data šŸ™‚#2019-10-2209:51henrik(Really good talk by the way)#2019-10-2214:19wilkerluciothanks šŸ™‚#2019-10-2117:25fjolneJoining @schmee on the question: is there a way to address ā€œN+1 problemā€ for SQL queries in Pathom? This one problem was pretty much the reason we migrated off ORM to writing raw SQL queries in HugSQL. Would be a life changer!#2019-10-2117:33kszabohttps://walkable.gitlab.io/ has this explicit goal#2019-10-2117:36fjolneThanks a lot! I somehow got misguided from the discussion above about Walkable.#2019-10-2117:38schmeefrom my experiments Walkable doesn’t solve this problem though#2019-10-2117:39schmeeI tried it with a 4 table join and it makes 4 separate queries#2019-10-2117:48fjolneWell, seems like it’s solved in a sense that it doesn’t make a request for every foreign key, which is a common problem for ORMs. Though I wonder how it could make an efficient join in app’s memory without loading the whole tables and without db indexes. #2019-10-2117:50schmeeahh! I don’t think my tables had foreign keys, I’ll add that and see how it affects the result#2019-10-2117:55wilkerlucioit is possible to work something using batch resolvers, for one depth level is simple, for more it may require some thinking#2019-10-2117:59kszaboif it doesn’t work out for you, try Hasura and integrate with it through the GraphQL integration of Pathom: https://github.com/walkable-server/walkable/issues/153#2019-10-2117:59kszaboAssuming you use Postgres#2019-10-2118:01schmee@U066U8JQJ is optimizations for SQL DBs an interesting use-case for Pathom or would you consider it ā€œout of scopeā€?#2019-10-2118:03wilkerlucioit is interesting for pathom, yes, I think walkable already does a nice job, what's missing is integrating that with the connect ecosystem, so they can compose on top of each other, I'm currently working to try to make those integrations easier, not just for sql, but graphql, pathom to pathom, and any other dynamic source, currently there are a ton of work to get those things "right"#2019-10-2118:03schmee> what’s missing is integrating that with the connect ecosystem, so they can compose on top of each other agree 100%, awesome that you are working on this šŸ™‚#2019-10-2118:04wilkerlucioI keep thinking if Pathom should move to a more "planned runner" thing, in the first versions it were purely recursive, meaning it decided what to do right before doing it, currently it does planning "per-attribute", which fixed a bunch of infinity loop cases, and more and more I see some benefits that could emerge by doing a "full planning" ahead of time#2019-10-2118:05wilkerlucioright now the algorithm has to reconcile a bunch of those, and when we try to integrate graph <-> graph things it ends up having to recompute paths a lot of times, also forces every resolver implementer to have to deal with that#2019-10-2118:06schmeeis it possible currently to check in a resolver what data was requested through it (through env or something else)? that would also help a lot when optimizing#2019-10-2118:55wilkerlucio@schmee not directly, what you have is just the parent-query, but I think that's something I can add#2019-10-2118:55wilkerlucioI'm currently changing the internal format of plan paths, its currently a vector with two items (which attribute its going for, and which resolver will call), I'm turning that into a map so more information can be made available on those#2019-10-2118:56fjolne@U08E8UGF7 thanks for suggestion! Unfortunately, we’re not on Postgres and there’s no way to migrate. I heard of Hasura, and your idea is really neat. Wonder if anyone done it that way, seems like a lot of indirection.#2019-10-2119:22schmee@U066U8JQJ cool, that would be really useful for me at least! šŸ™‚#2019-10-2213:36henrikNow that I have a connection to the GraphQL API up and running, I’m trying to figure out how to write a load! for the SearchResults component below.
(defsc Article
  [this {:zd.Article/keys [article_doi article_title]}]
  {:ident :zd.Article/article_doi
   :query [:zd.Article/article_doi
		   :zd.Article/article_title]}
  (div article_title))
(defsc SearchResults
  [this {:search-results/keys [id articles]}]
  {:ident :search-results/id
   :query [:search-results/id
		   {:search-results/articles (get-query Article)}]}
  (div
	(map article articles)))
Given that search queries look like this,
[{(:zd/search_articles_advanced_2
   {:n_results 1
	:filter_expr {:should [{:must [{:terminal_exprs {:matches_text "Water"}}]}]}})
  [:zd.Article/article_title
   :zd.Article/article_doi]}] 
And returns stuff like this:
{:zd/search_articles_advanced_2
 [{:zd.Article/article_title "Drinking Water",
   :zd.Article/article_doi "10.1177/1084822313481784"}]}
I suppose I want to write a resolver that is something like this (the only data of interest to the search are params, so I’m guessing it becomes global),
{::pc/output [{:search-results/articles […]}]}
` With … being some mysterious join into the world of the GraphQL API, and specifically the search_articles_advanced_2 entry point above (with the filter_expr being supplied as a param, I guess) so that I can ultimately do
(df/load! this [:search-results/id some-id] SearchResults
  {:target [:search-results/id some-id :articles]
   :filter-expr {:should {:must ["Water"]}}}
I’m not sure how to supply this glue—how would one do it?
#2019-10-2214:08wilkerlucio@henrik hello, one question, who is performing the actual search, the graphql API or something you are writing on pathom?#2019-10-2214:09henrikHey, the GraphQL endpoint is performing the search.#2019-10-2214:09henrikIt can basically be considered external, like your Youtube and SpaceX examples from the video.#2019-10-2214:10wilkerluciobecause, in this case, you may want to do that directly, make the components queries in a way that already aligns with them#2019-10-2214:11wilkerlucioone thing that's not obvious is how to set the parameters, IME I found that writing a fn that returns the query with the params is most straitforward way to do it, something like:#2019-10-2214:12wilkerlucio
(defn search-query [text-match]
 [{(:zd/search_articles_advanced_2
     {:n_results   1
      :filter_expr {:should [{:must [{:terminal_exprs {:matches_text text-match}}]}]}})
   [:zd.Article/article_title
    :zd.Article/article_doi]}])
#2019-10-2214:13wilkerlucioif you really want to rename the result somehow (instead of using the GraphQL names directly on your components), you can trigger a sub-query in a resolver, let me get you an example#2019-10-2214:16henrikI’m not sure how to write the components to line up with the GraphQL API. Is,
[{(:zd/search_articles_advanced_2
   {:n_results 1
    :filter_expr {:should [{:must [{:terminal_exprs {:matches_text "Water"}}]}]}})
  [:zd.Article/article_title
   :zd.Article/article_doi]}] 
even an acceptable query for a component in Fulcro, especially given that n_results and filter_expr are dynamic?
#2019-10-2214:18wilkerluciothe params part you need to fill during hte load! trigger (so you use the search-query fn as described before)#2019-10-2214:18wilkerluciothis is the example to run the sub-query internally:#2019-10-2214:18wilkerlucio
(pc/defresolver search-articles [env _]
  {::pc/output [{:search-results/articles 
                 […]}]
   ; this ::pc/params is more a documentation thing, doens't affect any runtime properties
   ::pc/params [:my-system/search-query]}
  (let [search-text (-> env :ast :params :my-system/search-query)
        ; if you are using async or parallel parsers, p/entity will return a core.async channel
        ; and you must wait for it
        search-result (p/entity env [{(:zd/search_articles_advanced_2
                                        {:n_results   1
                                         :filter_expr {:should [{:must [{:terminal_exprs {:matches_text search-text}}]}]}})
                                      [:zd.Article/article_title
                                       :zd.Article/article_doi]}])]
    ; write code to re-format search-results in whatever you want
    ...))
#2019-10-2214:19wilkerlucioto change the query, on the load! trigger you can use the property :update-query, this way you can replace the query entirely (there is where you would call search-query)
#2019-10-2214:19wilkerluciomakes sense?#2019-10-2214:21henrikI’m slowly wrapping my head around it. The sub-query solution seems to assume that
[:zd.Article/article_title
 :zd.Article/article_doi]
Are hard-coded, whereas I would very much like them to flow from the fact that they were selected by the Fulcro component.
#2019-10-2214:22wilkerlucioyeah, considering that, I would suggest you just use the final GraphQL names on the component, I can do a quick write on how I imagine it, once sec šŸ™‚#2019-10-2214:24wilkerluciowhat about this?#2019-10-2214:24wilkerlucio
(defsc Article
  [this {:zd.Article/keys [article_doi article_title]}]
  {:ident :zd.Article/article_doi
   :query [:zd.Article/article_doi
           :zd.Article/article_title]}
  (div article_title))

(defn load-search-results [component query-text]
  (df/load-field component :zd/search_articles_advanced_2
    {:params {:n_results   1
              :filter_expr {:should [{:must [{:terminal_exprs {:matches_text query-text}}]}]}}}))

(defsc SearchResults
  [this {:search-results/keys [id articles]}]
  {:ident :search-results/id
   :query [:search-results/id
           {:zd/search_articles_advanced_2 (get-query Article)}]}
  (div
    (dom/button {:onClick #(load-search-results this "search")} "Search!")
    (mapv article articles)))
#2019-10-2214:26henrikProcessing… Please wait. ā³#2019-10-2214:27henrikSeems far better by the look of it.#2019-10-2214:39henrikRegarding load-field, the search results component won’t own the search function. It must come from a parent of both the results component and the search field/input component. How would you deal with this? Extract it from the component registry?#2019-10-2214:42henrikOther than that conundrum, I like this solution. The component query very explicitly shows directly how it depends on the search API, which is nice. The less impedance mismatch, the better.#2019-10-2214:44henrikI guess that a solution to the above is to just store the results in the top-level component rather than in the sub-component of SearchResults.#2019-10-2214:50wilkerlucio@henrik didn't get the load-field part, the data will live inside of the SearchResults ident in this case, would be good if you have some id for the search results component, that can be a random generated uuid#2019-10-2214:51wilkerlucioits ok for pathom to do that query from inside of an ident, pathom should generate the appropriated GraphQL query in this case, even so it not top level#2019-10-2214:52wilkerluciothere is another way to trigger that load, maybe this:#2019-10-2214:53wilkerlucio
(defn load-search-results [component query-text]
  (df/load component :zd/search_articles_advanced_2
    {:target (conj (fp/get-ident component) :zd/search_articles_advanced_2)
     :params {:n_results   1
              :filter_expr {:should [{:must [{:terminal_exprs {:matches_text query-text}}]}]}}}))
#2019-10-2214:54wilkerlucio(I'm not testing any of this, so some details may need adjust)#2019-10-2215:33henrikSo, looking at the query that load! sends to the Pathom parser, it ends up being:
[({[:search-results/id :an-id]
   [:search-results/id
    {:zd/search_articles_advanced_2
     [:zd.Article/article_doi :zd.Article/article_title]}]}
  {:n_results 1,
   :filter_expr
   {:should [{:must [{:terminal_exprs {:matches_text "hello"}}]}]}})]
#2019-10-2215:35henrik(with some dummy data, just to see what it says)#2019-10-2215:36henrikSorry, I may have done something stupid.#2019-10-2215:39henrikAh, yes, here’s the missing ticket:
(df/load app :zd/search_articles_advanced_2 Article …)
Give it Article, or otherwise it doesn’t select any fields in the query to GraphQL.
#2019-10-2215:41henrikWell, that certainly goes out and loads some data! Awesome!#2019-10-2215:43henrik(Since I haven’t actually wired the components into the UI yet, I wanted to try it out on app before assuming that I’ve understood more than I have)#2019-10-2216:20henrikI’m going to have to buy you a beer next time I’m in Brazil.#2019-10-2218:17henrikYou think I’m joking, but my wife is a Paulista 😁#2019-10-2218:35wilkerlucionice! I'll be glad to have a beer with you! do you have plans to visit here soon?#2019-10-2309:45henrikWe were going to go at the end of the year, but a surprise happened and all her family is coming to Stockholm instead! So preliminarily, early next year.#2019-10-2219:39twicebakedSilly question, is there any way to rename a resolver with a single input defined with defresolver or is it always auto-named? If so, what's the best practice? I of course can rename the input which changes the resolver name, but it's a bit odd destructuring a param that's really just there to change the name. Maybe using aliases/alias resolvers is possible?#2019-10-2219:49wilkerlucio@twicebaked is worth noticing that those helpers just return maps, so the easiest way to rename is doing: (assoc (pc/single-attr-resolver ...) ::pc/sym 'my/new-name)#2019-10-2219:51twicebakedOK, yeah I figured that manipulating pc/sym waas what would do it. Just wondered if there was a simpler way maybe via the macro itself but didn't see anything obvious in the source right away. Thanks.#2019-10-2220:09wilkerluciono built-in helper, but you can make your own šŸ™‚#2019-10-2220:10wilkerluciothis project is serious about keeping things as stable as possible (include keywords), so you can count that ::pc/sym is not something that will change#2019-10-2309:49henrikDo I have to do something in particular to support :com.wsscode.pathom.graphql/aliased fields (add them manually to the index)? The query finishes the roundtrip, but the field shows up unaliased, and resolved to :com.wsscode.pathom.core/not-found. Checking the query with query->graphql, the query does resolve properly, with proper GraphQL syntax produced for the aliased field.#2019-10-2310:03henrikThis is the expression:
(:zd.Article/article_has_attribute
 {::pg/alias "has_fulltext_edn"
  :attribute_name "article_fulltext__hiccup_edn"})
The result:
{:zd.Article/article_has_attribute :com.wsscode.pathom.core/not-found
 …}
Dropping the alias results in:
{:zd.Article/article_has_attribute true
 …}
#2019-10-2312:05wilkerlucio@henrik hello, this is a current limitation when doing graph/graph communication, the way pathom figures the fields to send to graphql is by looking on its own query, but, it only looks at things that are directly visible (in your query) without looking for those things dependencies, in your case, the alias makes a dependency on a "hidden" field, and that's not picked up by pathom currently, if you query using both fields (original graphql name + alias) it will work#2019-10-2312:09henrikGotcha, so theoretically, the short-term fix should be
[:has_fulltext_edn
 '(:zd.Article/article_has_attribute
   {::pg/alias "has_fulltext_edn"
    :attribute_name "article_fulltext__hiccup_edn"})]
#2019-10-2312:10henrikThough I seem to end up with another attribute that is not found.#2019-10-2312:11wilkerluciooh, I may had got your question wrong, what kind of alias is that (I was confusing with connect aliases)#2019-10-2312:12henrikAha, sorry, probably my poor explanation. This is about aliasing GraphQL fields.#2019-10-2312:12wilkerluciocheck it out here: https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/graphql/edn-to-gql.html#2019-10-2312:12wilkerlucioclick on the Aliases example#2019-10-2312:13henrikThat’s where I started, yes. This doesn’t in practice work when I do it through Fulcro Inspector.#2019-10-2312:13wilkerluciobut I don't remember if that works with the connect integration, if what you want is to have a diffrent name on the EQL output, you may want to do this:#2019-10-2312:14wilkerlucio
(:zd.Article/article_has_attribute
  {:pathom/as :my/new-output-name
   :attribute_name "article_fulltext__hiccup_edn"})
#2019-10-2312:15wilkerluciobut in Fulcro that can get confusing, because the query will not match the output#2019-10-2312:15wilkerluciocan you tell more about your use case for doing the aliasing?#2019-10-2312:17henrikSure, the API defines article_has_attribute as a way to check whether any particular field is present or absent on a particular article. Since some parts of an articule can weigh up to 10mb, this is necessary in order to know whether to show a button for downloading the full text of the article (for example), without actually retrieving the entire fulltext before the button is clicked.#2019-10-2312:18henrik:pathom/as seems to produce the same result. Hang on, I’ll show the queries.#2019-10-2312:22henrik(As such, article_has_attribute can show up multiple times in a query, necessitating aliasing for each)#2019-10-2312:59wilkerluciogotcha, makes sense, I'm not sure if that's currently supported, but I understand the need, today I'm pretty busy but I can take a closer look on how we can support that tomorrow#2019-10-2313:23henrikThanks a lot! Perhaps EQL should have a general aliasing syntax, rather than being Pathom-specific? It seems like it could be regarded as a structural concern of the syntax, rather than an application feature (although a converter from EQL->something else would still have to know how to deal with it in the target language, of course).#2019-10-2313:42henrikMaybe it’s a bit wonky, maybe it interferes with other metadata concerns in EQL, but maybe something like:
{^{:alias :bar} 
 (:foo {:with "params"})
 [:something]}
#2019-10-2314:23wilkerluciowell, I like to try to thing of changes that has the least impact on the current things, that considered I think we maybe want something in the opposite direction, like: (:aliased-name {:pathom/source :original-name})#2019-10-2314:23wilkerluciobecause doing that would just work with Fulcro and other parts of the system that expect the final value to be the attribute source in the query, makes sense?#2019-10-2314:26henrikYes, that’s fair enough! It certainly looks better, and vaguely mirrors the syntax they choose for GraphQL (not that GraphQL necessarily is the gold standard, of course)#2019-10-2312:07wilkerlucioso, computing all sub-variations can get expensive quickly, I'm currently working on a new query planner that I hope will make this better, because in the current state its hard to get "dynamic resolvers" right because of those implicit dependencies#2019-10-2312:18henrikWithout aliasing#2019-10-2312:19henrikWith ::pg/alias#2019-10-2312:19henrikWith :pathom/as#2019-10-2808:21henrik@wilkerlucio What do you think of this (hypothetically)? Query:
[{(:zd/articles_selection_2
   {:filter_expr 'filter-expr})
  [{(:zd.ArticlesSelection/articles_sel_top_subject_categories_distribution
     {:n_results 'n-results})
    […]}
   {(:zd.ArticlesSelection/articles_sel_top_journals_distributions
     {:n_results 'n-results})
    […]}]}]
Then in load!,
(defn load-filters [filter-bar-id filter-expr n-results]
  (load! app :zd/articles_selection_2 Filters
    {:params {'filter-expr filter-expr
              'n-results n-results}}))
The idea would be to have Pathom slot in the values from params in any sub- queries. It would 1) make the composition of queries that require multiple levels of arguments more straightforward, 2) be quite easy to throw meaningful exceptions or warnings when values for parameters are missing (on load!), 3) minimize the amount of data on the wire when Pathom is running on the backend, and arguments are repeated in the query, and 4) be more or less trivial to pass on to GraphQL (if it is the ultimate destination) as a query and variables combo.
#2019-10-2817:43wilkerlucioconsidering that this is more a problem of building the query than runnign it, to me this feels more like a client issue (on building the query), given queries compose and can go very large I would avoid trying to do a replace like this, but you can totally do it locally and use as a :update-query, so you can implement this "templaing", but to be honest, one you do that, you can realize havign a function with params do pretty much the same, makes sense?#2019-11-0108:21henrikWell, that’s mostly an incidental side-effect. What it really would do is deduplicate repeating args. Consider, for example, the argument,
{:should
 [{:must
   [{:terminal_exprs {:matches_text "water"}}
    {:terminal_exprs {:author_name_matches "Andersson"}}],
   :filter
   [{:terminal_exprs
     {:article_subject_categories_match "Chemistry"}}
    {:terminal_exprs {:is_oa_equals true}}]}
  {:must [{:terminal_exprs {:matches_text "soil"}}],
   :filter
   [{:terminal_exprs
     {:article_subject_categories_match
      "Agricultural and Biological Sciences"}}
    {:terminal_exprs
     {:article_subject_categories_match
      "Agronomy and Crop Science"}}
    {:terminal_exprs
     {:publisher_name_matches "Springer Nature"}}]}]}
This is a very average argument in our case, neither small nor large. This can be repeated up to three times in a single query (often dwarfing the actual query itself). Transit will shrink this a tiny bit, but not meaningfully so. This’ll be repeated three times frontend to backend (where Transit will help a little), and then three times again backend to API (which is GraphQL, and Transit won’t be there to help at all). The primary attraction of the above is that it could cut a lot of unnecessary network overhead.
#2019-10-2817:39BrianI'm getting an "invalid expression" error when running the following mutation and I can't figure out why:
(pc/defmutation add-hash [env artifact]
  {::pc/sym `ab.threat-intelligence.artifact/add-hash
   ::pc/params [:artifact/note]
   ::pc/output [:artifact/note2]}
  {:artifact/note2 (get artifact :artifact/note)})
(<!! (parser {} [{(ab.threat-intelligence.artifact/add-hash {:artifact/note "hello"}) [:artifact/note2]}]))

#2019-10-2817:42wilkerlucioyou have to quote the symbol name when you call the expression:
(<!! (parser {} [{(list 'ab.threat-intelligence.artifact/add-hash {:artifact/note "hello"}) [:artifact/note2]}]))
#2019-10-2817:49BrianLets GOOOOO thank you!#2019-10-2916:05eoliphanthey @wilkerlucio finally got some time to play around with pathom-datomic, great stuff šŸ™‚, I’m running into a few cloud v on-prem issues. it looks like say say the query-entities helper is using the single tuple find-spec, where unfortuantely cloud only supports the relation spec. I’ve stared playing around with the source a bit to see what needs to be done#2019-10-2918:06wilkerlucio@eoliphant thanks! sure, I didn't tried the cloud, but I remember someone else also pointing that out, if you can figure I'll be happy to take a PR to fix it#2019-10-2918:06eoliphantwiill do šŸ™‚#2019-10-3020:40BrianSo I have this mutation:
(pc/defmutation add-artifact [env artifact]
  {::pc/sym `r9b.threat-intelligence.artifact/add-artifact
   ::pc/params [::note ::hash/sha-256]
   ::pc/output [:artifact/note]}
  ...)
How can I access the params? I've noticed that there can be any keys inside artifact and I want to remove all those that are not in the ::pc/params so that I can then reduce over artifact to do something meaningful. Is that possible?
#2019-10-3023:11wilkerlucio@brian.rogers hello, one way to do it is writing a ::pc/transform, using that you can wrap the mutation fn and do anything you want (like stripping params), a example:#2019-10-3023:11wilkerlucio
(defn filter-declared-params-transform
  [{::pc/keys [mutate params] :as mutation}]
  (cond-> mutation
    (and mutate params)
    (assoc
      ::pc/mutate
      (fn [env p]
        (mutate env (select-keys p params))))))

(pc/defmutation add-artifact [env artifact]
  {::pc/sym       `r9b.threat-intelligence.artifact/add-artifact
   ::pc/params    [::note ::hash/sha-256]
   ::pc/output    [:artifact/note]
   ::pc/transform filter-declared-params-transform}
  ...)
#2019-10-3023:13wilkerluciothen, if you wanna use it a lot, you can create your own defmutation that always adds it, or, considering that mutations are just maps, you can do some pre-processing on them before adding to the index (or maybe change the index after its ready, the mutation code fn also lives there)#2019-10-3023:39eoliphantHEy I’m getting a ā€œSeems like the index is not available.ā€ in the inspector, I have the recommended index resolver in my parser config. any suggestions on a query to say run manually in the repl to diagnose the problem?#2019-10-3100:13eoliphantdid some more digging, looks like it might be the datomic connect plugin. Took it out, it worked fine started adding it back in
java.lang.RuntimeException: java.lang.Exception: Not supported: class com.wsscode.pathom.connect.datomic$index_schema$fn__34159
	at com.cognitect.transit.impl.WriterFactory$1.write(WriterFactory.java:65)
#2019-10-3103:58souenzzo
(pc/defresolver index-explorer [{::pc/keys [indexes]} _]
                {::pc/input  #{:com.wsscode.pathom.viz.index-explorer/id}
                 ::pc/output [:com.wsscode.pathom.viz.index-explorer/index]}
  {:com.wsscode.pathom.viz.index-explorer/index (p/transduce-maps
                                                  (remove (comp #{::pc/resolve ::pc/mutate}
                                                                key))
                                                  indexes)})

I use this resolver
#2019-10-3109:59henrikFWIW, I’m not getting indices either. Q:
[{[:com.wsscode.pathom.viz.index-explorer/id
   [:fulcro.inspect.core/app-uuid
    #uuid "85b4ac42-f4bb-48b8-92bb-b9c3a769d843"
    :remote]]
  [:com.wsscode.pathom.viz.index-explorer/id
   :com.wsscode.pathom.viz.index-explorer/index]}]
A:
{[:com.wsscode.pathom.viz.index-explorer/id
  [:fulcro.inspect.core/app-uuid
   #uuid "85b4ac42-f4bb-48b8-92bb-b9c3a769d843"
   :remote]]
 {:ui/fetch-state {:fulcro.client.impl.data-fetch/type :not-found}}}
#2019-10-3109:59henrikThis is with the above index resolver.#2019-10-3101:03wilkerlucio@eoliphant I guess you are hitting this: https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/exploration.html#_fixing_transit_encoding_issues#2019-10-3101:28eoliphantah ok#2019-10-3101:30eoliphantso after creating the writer#2019-10-3101:31eoliphantdoes it need to be jacked into the parser config?#2019-10-3102:18wilkerluciothis has nothing to do with pathom really, its a issue when you try to send things over transit that are not encodable by default (like fns)#2019-10-3102:18wilkerluciomy tutorial is to add a default handler so you just send a bogus data, but doens't break the encoding#2019-10-3111:04henrikIs there any way to…
;; Transform…
{:a {:b [d1 d2 d3 … dn]
     :c [d1 d2 d3 … dn]}}

;; Into…
{:a {:b {:>/group [d1 d2 d3 … dn]}
     :c {:>/group [d1 d2 d3 … dn]}}}
… using placeholders?
#2019-10-3114:44wilkerlucio@henrik I don't understand what you are trying to do, can you tell more about your use case?#2019-10-3114:52henrikSure, :b and :c (and more) return children on the exact same format. There has to be a group component between :a and the ds (I can’t render them directly in the component that has a as query). I could make different components for b and c etc., but that would be redundant. The other alternative is to destructure both b and c in a unified group component, then (or b c) to render the children. Alternatively, destructure :b and :c in a, then send the children as props to the group component (which feels a bit funky, I’m conditioned to want props to be a map). Or, I effectively rename :b and :c to something unified that the group component will understand, for which one alternative might be to scope the ds under some well known key.#2019-10-3114:55henrikI’m dealing with a deep map with no natural idents as far as the eye can see.#2019-10-3115:04henrik
;; Data
{:a {:b [d1 d2 d3 … dn]
     :c [d1 d2 d3 … dn]}}

;; One component for `a`, one for the `b`/`c` level, one for each `d`

;; Alternative 1
(defsc ComponentForA [this {:keys [b c]}]
  (ui-group-comp b)
  (ui-group-comp c))

(defsc GroupCompForBorC [this props]
  ;; Props will be a vector
  (map ui-d-comp props)) 


;; Alternative 2
(defsc ComponentForA [this {:a/keys [b c] :as props}]
  (ui-group-comp (select-keys props [b]))
  (ui-group-comp (select-keys props [c])))

(defsc GroupCompForBorC [this {:keys [b c]}]
  ;; Props will be a  map of either 
  ;; {:b […]} or {:c […]}
  (map ui-d-comp (or b c)))


;; Ideally
(defsc ComponentForA [this props]
  ??)

(defsc GroupComp [this {:keys [group-of-ds]}]
  (map ui-d-comp group-of-ds))
#2019-10-3116:21henrikForget all of the above, it turns out I’m trying to optimize for a pointless scenario.#2019-10-3117:06wilkerlucio@henrik glad you reach a point, IME is always good to have some ident to any thing you reference, I did bit myself multiple times when I decided to not have one#2019-10-3117:16henrikIt’s not really up to me to decide I’m afraid, I have to work with the data as returned by the API šŸ™‚ There are no idents, so I can’t pretend like there are.#2019-10-3117:19henrikEssentially, it’s
{:search-filters 
 {:categories [{:title "Biology"
                :count 3214}
               {:title "Chemistry"
                :count 89}
               …]
  :publishers [{:title "Springer"
                :count 212}
               …]
  :journals [{:title "Nature"
              :count 2131}]}}
#2019-10-3117:19henrikBut with a more complicated naming structure. Not much in there to grab onto as an ident.#2019-10-3117:33wilkerlucioyeah, that always sucks... (when you can't control it, but must use it)#2019-11-0109:57henrikFor fun, I created a toy resolver to understand how batching works with resolvers. This led to some surprising results, with the batching taking quite a bit longer to run than the serial one. It seems that this is due to it incrementally running the resolver with one new input added each time. Have I missed something crucial here? (It’s using Aleph, so http/get returns a Manifold)#2019-11-0110:35henrikNevermind, I just discovered the difference between parser and parallel-parser šŸ™‚ Initially, I thought it redundant, since the web server is parallel anyway. Turns out it has more implications than that.#2019-11-0118:18BrianHey @wilkerlucio I'm having some problems here. I'm hoping to be able to read a mutation query off the wire from an API and then throw it into pathom. A call like this works perfectly fine: (<!! (parser {} '[{(hello {}) [:greeting]}])) however when I remove the single quote yielding (<!! (parser {} [{(hello {}) [:greeting]}])) I get an "invalid expression" error. This matters because when I read my queries off the wire using code like this:
(let [query (read-query-off-the-wire)]
    (<!! (parser {} (quote query))))
it does not evaluate the contents of the query variable. So now I'm faced with this weird problem where pathom wants an unevaluated expression, and I need to evaluate my variable within a quote function which stops me from being able to evaluate it. What can I do?
#2019-11-0118:24wilkerluciohello @brian.rogers , the reason you need to quote when you write the code like that is that otherwise it tries to run it like a fn, which is not what you need, you need a list with symbol#2019-11-0118:25wilkerlucioif you get something that is already that literal list, you dont need to quote, so you can just forward it, makes sense?#2019-11-0119:01BrianThank you @wilkerlucio!#2019-11-0622:19daniel.spanielHas anyone ever seen situation where pathom query freezes datomic cloud. I am finding that certain queries ( issued from fulcro and passed on to pathom , which passes them on to datomic cloud just freeze. There are no errors just browser hangs waiting for request to ( in this case -> never ) finish We kinda hurting our brains to figure it out.#2019-11-0622:21daniel.spanielThis is only when running locally connecting to datomic. In production its fine.#2019-11-0701:30wilkerluciohello @dansudol, are you using the pathom-datomic package, or doing the integration with regular resolver?#2019-11-0701:34daniel.spanielhonestly not 100% sure Wilker ..#2019-11-0701:34daniel.spanielbut I found the issue was with a particular join query#2019-11-0701:34daniel.spanielit seems to work fine in production but not locally#2019-11-0701:35daniel.spanielit is truely bizzarre and i think you might have more insight than i#2019-11-0702:27wilkerluciocan you show the query that you see trouble? did you tried to debug using the pathom trace?#2019-11-0704:38cjmurphy@dansudol With Pathom I have seen execution on the server just halting, with no error being reported. For example I recently had this problem when server code did a select-keys with the 2nd arg being a keyword rather than a vector as it is supposed to be. I only got to see the error by calling the code directly, outside of Pathom. The issue is that core.async errors are not reported to the thread/REPL you are running in. If Pathom could be run without using core.async I believe this error message (for bad select-keys call) would have been seen in the REPL. I think that this kind of problem will be fixed when spec becomes a part of Clojure, as ordinary functions will then be checked. At the moment I can see that guardrails/Ghostwheel issues are reported by expound - so spec works with Pathom. Another thing is that just regular asserts will also not show up, although you can get around this by having your own assert macro that writes the message and stack trace to a file before throwing the AssertionError.#2019-11-0714:59wilkerluciofor debugging this case of thing, I suggest you try to use the serial parser, I'm actually starting to think the serial is a better general recommendation than the parallel, its simpler to understand and debug, and I believe most people are not taking that much out of the parallel (you need to have a fair amount of parallelism opportunity to get significant gains with that)#2019-11-0715:00wilkerlucioat Nubank we have extra setup to get errors reported in a more systematic way, you can use the ::p/process-error to setup a custom error handler, that works with both serial and parallel, from that I suggest you can log the errors in some source you can lookup later#2019-11-0716:08daniel.spanielgood suggest about logging errors chris and i am trying serial parser now#2019-11-0716:09daniel.spanielalso I have set up what I thought was error reporting this way#2019-11-0716:09daniel.spaniel
::p/plugins    [(pc/connect-plugin {::pc/register (vec (vals @pathom-registry))})
                                       (p/env-wrap-plugin
                                         (fn [{:keys [conn] :as env}]
                                           (merge env {:config
                                                       config
                                                       ::p/process-error
                                                       (fn [_ err]
                                                         ; print stack trace
                                                         (.printStackTrace err)

                                                         ; return error str
                                                         (p/error-str err))})))
                                       (preprocess-parser-plugin log-requests)
                                       (p/post-process-parser-plugin p/elide-not-found)


                                       ;p/request-cache-plugin
                                       p/error-handler-plugin
                                       p/trace-plugin]
#2019-11-0716:09daniel.spanielbut it is not throwing out any errors#2019-11-0716:11daniel.spanielactually when switching to serial parser i got an error#2019-11-0716:11daniel.spaniel
java.lang.IllegalArgumentException: No implementation of method: :take! of protocol: #'clojure.core.async.impl.protocols/ReadPort found for class: clojure.lang.PersistentArrayMap
#2019-11-0716:11daniel.spanielso i switching back to parralel#2019-11-0716:40wilkerlucio@dansudol about the switch, you also need to switch the reader, instead of pc/parallel-reader you should use pc/reader2#2019-11-0716:40wilkerlucioalso the mutation fn must be changed, from pc/mutate-async to pc/mutate#2019-11-0717:21daniel.spanielWow .. @U066U8JQJ thank goodness you reminded me about those 2 methods ( even without looking at my setup ) that is what I had#2019-11-0717:21daniel.spanielI also had this#2019-11-0717:21daniel.spaniel
(fn wrapped-parser [env tx]
      (async/<!! (real-parser env (if trace?
                                    (conj tx :com.wsscode.pathom/trace)
                                    tx))))
#2019-11-0717:22wilkerlucioyeah, that needs to change, the serial returns a regular map, simpler stuff šŸ™‚#2019-11-0717:22daniel.spanielso you can see why it was barfing .. gosh .. amazing .. but your idea helped me out#2019-11-0717:22daniel.spanieli changed to#2019-11-0717:22daniel.spaniel
(fn wrapped-parser [env tx]
      (real-parser env (if trace?
                         (conj tx :com.wsscode.pathom/trace)
                         tx)))
#2019-11-0717:23daniel.spanieleverything works now .. amazing#2019-11-0717:28wilkerlucioglad to hear šŸ™‚#2019-11-1217:30mruzekwHi there, I’m building a backend with pathom in CLJS. I’m following the Fulcro guide and I’m at the point where I’m playing with parsers in the REPL.#2019-11-1217:30mruzekwhttp://book.fulcrologic.com/fulcro3/#_parsing_queries#2019-11-1217:30mruzekwHowever, when I pull my parser namespace into the REPL and call api -parser it gives me this error:
TypeError: Cannot read property 'ReadPort' of undefined
    at $wsscode$common$async_cljs$chan_QMARK_ [as chan_QMARK_] (/Users/mruzekw/Code/serverlesstest/myapp/.shadow-cljs/builds/node-repl/dev/out/cljs-runtime/com/wsscode/common/async_cljs.cljs:7:3)
#2019-11-1217:31mruzekwCan the ns be loaded and the parser without a running server?#2019-11-1217:31mruzekwI’m sort of lost where to go from here#2019-11-1217:31mruzekwFor the record I’m trying to run:
(api-parser [{[:person/id 1] [:person/name]}])
#2019-11-1217:32mruzekwAnd the parser and resolvers are defined exactly as in the Fulcro section#2019-11-1217:36souenzzo@mruzekw can you share the full stacktrace? ( *e on repl )#2019-11-1217:37mruzekwSure#2019-11-1217:37mruzekw
TypeError: Cannot read property 'ReadPort' of undefined
    at $wsscode$common$async_cljs$chan_QMARK_ [as chan_QMARK_] (/Users/mruzekw/Code/serverlesstest/myapp/.shadow-cljs/builds/node-repl/dev/out/cljs-runtime/com/wsscode/common/async_cljs.cljs:7:3)
    at com$wsscode$pathom$core$wrap_parser_exception_$_wrap_parser_exception_internal (/Users/mruzekw/Code/serverlesstest/myapp/.shadow-cljs/builds/node-repl/dev/out/cljs-runtime/com/wsscode/pathom/core.cljc:830:7)
    at /Users/mruzekw/Code/serverlesstest/myapp/.shadow-cljs/builds/node-repl/dev/out/cljs-runtime/com/wsscode/pathom/connect.cljc:1695:13
    at $wsscode$pathom$core$wrap_normalize_env_internal__3 [as cljs$core$IFn$_invoke$arity$3] (/Users/mruzekw/Code/serverlesstest/myapp/.shadow-cljs/builds/node-repl/dev/out/cljs-runtime/com/wsscode/pathom/core.cljc:1009:7)
    at $wsscode$pathom$core$wrap_normalize_env_internal__2 [as cljs$core$IFn$_invoke$arity$2] (/Users/mruzekw/Code/serverlesstest/myapp/.shadow-cljs/builds/node-repl/dev/out/cljs-runtime/com/wsscode/pathom/core.cljc:1006:4)
    at myapp$backend$parser$api_parser (/Users/mruzekw/Code/serverlesstest/myapp/.shadow-cljs/builds/node-repl/dev/out/cljs-runtime/myapp/backend/parser.cljs:47:3)
    at cljsEval (<eval>:1:41)
    at global.SHADOW_NODE_EVAL ([stdin]:105:10)
    at Object.shadow$cljs$devtools$client$node$node_eval [as node_eval] (/Users/mruzekw/Code/serverlesstest/myapp/.shadow-cljs/builds/node-repl/dev/out/cljs-runtime/shadow/cljs/devtools/client/node.cljs:24:1)
    at /Users/mruzekw/Code/serverlesstest/myapp/.shadow-cljs/builds/node-repl/dev/out/cljs-runtime/shadow/cljs/devtools/client/node.cljs:49:13
#2019-11-1217:41souenzzolooks like that you are mixing -async things with non-async things . But I can't understand how.. You are using some p/async-parser or pc/mutate-async or some -async reader?#2019-11-1217:42mruzekwThe parser is defined as:
(def pathom-parser
  (p/parser {::p/env     {::p/reader                 [p/map-reader
                                                      pc/reader2
                                                      pc/ident-reader
                                                      pc/index-reader]
                          ::pc/mutation-join-globals [:tempids]}
             ::p/mutate  pc/mutate
             ::p/plugins [(pc/connect-plugin {::pc/register resolvers})
                          p/error-handler-plugin]}))
#2019-11-1217:55wilkerlucioFulcro by default assumes you are using the parallel parser, the parallel / async return channels, but the normal one doesn't, look for a place with <!! and remove it, so it doesn't try to read from a map šŸ™‚#2019-11-1217:55wilkerluciothat's my guess ,but let me know if its something different#2019-11-1217:55wilkerluciohow are you integrating this parser? this is a cljs parser, right?#2019-11-1217:56mruzekwYep, CLJS. Plan on using it in an AWS Lambda#2019-11-1217:56wilkerlucio(for cljs you probably want the async-parser, otherwise you wont be able to do anything async, like http requests)#2019-11-1217:56wilkerlucioah, on lamba, ok, this way you may be ok with serial, hehe#2019-11-1217:57wilkerluciohow did you setup the fulcro remote for it?#2019-11-1217:57souenzzocan't reproduce. https://gist.github.com/souenzzo/58cf26321fc2ab91fde9dbcef9b3781a Should be something "external" (as fulcro wraper)#2019-11-1217:57mruzekwRight now I’m just hard coding some maps and resolving from there#2019-11-1217:58mruzekwI’m not really in the Fulcro part of the set up just yet#2019-11-1217:58mruzekwJust trying to play with EQL, Pathom, and resolvers#2019-11-1217:59wilkerluciostrange, can you post the full code of what you trying on a gist?#2019-11-1218:00mruzekwSure#2019-11-1218:00mruzekwhttps://gist.github.com/mruzekw/4a5aff0a11273d37dc00d4d6e5084624#2019-11-1218:01mruzekwI’m just loading that ns right now and trying to run api-parser#2019-11-1218:01wilkerluciowhy do you have 2 parsers?#2019-11-1218:01mruzekwI was trying either#2019-11-1218:03mruzekwNeither of them work#2019-11-1218:03wilkerluciothe parallel will return a channel#2019-11-1218:04wilkerluciobut I'm finding very strange that error you got#2019-11-1218:04wilkerluciofor read port#2019-11-1218:04wilkerluciothat's nothing much else going on around?#2019-11-1218:04wilkerluciobut also, that setup on the regular parser seems old#2019-11-1218:04wilkerluciotry this:#2019-11-1218:05wilkerlucio
(def pathom-parser
  (p/parser {::p/env                  {::p/reader [p/map-reader
                                                   pc/reader2
                                                   pc/open-ident-reader
                                                   p/env-placeholder-reader]}
             ::p/placeholder-prefixes #{">"}
             ::p/mutate               pc/mutate
             ::p/plugins              [(pc/connect-plugin {::pc/register resolvers})
                                       p/error-handler-plugin
                                       p/trace-plugin]}))
#2019-11-1218:07wilkerlucioone detail, on your parallel. you were using a wrong variable on the resolvers#2019-11-1218:07wilkerlucioalso, you don't need double list here: (def resolvers [[person-resolver list-resolver]])#2019-11-1218:07wilkerlucioit gets flatten, the reason to support vectors is jjust to make composition easy#2019-11-1218:07wilkerluciobut in pratice its a flat list#2019-11-1218:08mruzekwSure#2019-11-1218:08mruzekwDoesn’t seem to work either#2019-11-1218:22mruzekwSo p/parser is just a sync parser and should just return the result, right? No channel needed?#2019-11-1218:23mruzekwWhy would it be touching async stuff then?#2019-11-1218:24wilkerluciothat's what is bugging me, makes no sense šŸ˜›#2019-11-1218:24wilkerlucioso, I wrote a small demo for you: https://github.com/wilkerlucio/pathom-cljs-demo#2019-11-1218:24wilkerluciothis repository is working, you can clone and run shadow-cljs watch app#2019-11-1218:24wilkerlucioit logs a parser result to the console#2019-11-1218:25wilkerlucioI hope you may find a difference with your setup#2019-11-1218:26wilkerlucioconsole here#2019-11-1218:27mruzekwThanks, I’ll give it a look.#2019-11-1218:27mruzekwUsing a Node env wouldn’t make a difference, would it?#2019-11-1218:28wilkerlucioit should be the same#2019-11-1218:28mruzekwOkay#2019-11-1218:28mruzekwIt does work with your repo#2019-11-1218:28mruzekwI’ll see if I can reconcile with my setup#2019-11-1218:28mruzekwThank you#2019-11-1218:33mruzekwOkay it seems to be working now.#2019-11-1218:33mruzekwThank you both!#2019-11-1218:34mruzekwI’m not sure if my shadow-cljs compiler was watching for changes. Thus the code you originally gave me probably worked, but wasn’t loaded when I reloaded the ns in the REPL#2019-11-1218:52wilkerlucioand state crashes the day again, hehe#2019-11-1218:52wilkerlucioyou should check if that behavior was expected at #shadow-cljs#2019-11-1218:54mruzekwkk, thanks#2019-11-1218:36mruzekwWhat’s the advantage of Pathom over plain GraphQL other than EDN and EQL?#2019-11-1218:38mruzekwIf I were to argue to non-Clojure developers why this is valuable, what would I say?#2019-11-1218:39mruzekwIs it most valuable in the context of Fulcro?#2019-11-1218:55wilkerlucio@mruzekw to me, when comparing Pathom with GraphQL, the main difference is on the modeling, GraphQL uses a strict type schema, were types must defined ahead of time, while pathom embraces a very flexible property based modeling, where the central pieces are the properties, not the entities. this has a bunch of implications down the road, one example is when you try to combine multiple graphs, in the GraphQL case, its a pain in the ass, because you can't extend any of the types, also, because of the lack of namespaces, there is a good change you end up with conflicting types, they have workarounds, but IMO they fail to address the main issue here, that is, you need some way to talk about information that's not ambiguous.#2019-11-1218:58wilkerlucioin this sense, pathom things more like RDF, and encourage the use of long names (eg: :youtube.video/id instead of :id), those names enable multiple systems to live in the same space, this also enables that you make connections around those names. this way pathom makes easier to combine different graphs, in my work, our system deals with tons of REST endpoints, plus 2 different GraphQL sources, and you can get data from all of them in a single query, I find this property beautiful šŸ™‚#2019-11-1218:59wilkerlucioand the trick for the GraphQL was simple, we just prefix all the names imported, so they wont collide#2019-11-1219:00wilkerlucioand when you have the base on properties, you can have style of resolvers you are seeing in Connect, that effectivelly kills the need for a controler, the index can do the composition for you#2019-11-1219:00wilkerluciodoes some of that makes sense to you?#2019-11-1219:36mruzekwFrom a high-level it does. I haven’t written a plain GraphQL resolver/server to completely understand the index part yet#2019-11-1219:37mruzekwI like the idea of property-based modeling, seems more dynamic#2019-11-2018:32pithylessHas anyone got a working example of aleph + muuntaja + pathom?#2019-11-2018:32pithylessPlaying around with Index Explorer, but I'm hitting an encoding error that looks eerily similar to this issue: https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/exploration.html#_fixing_transit_encoding_issues#2019-11-2020:07pithylessI've temporarily resolved this via a custom transit writer: https://clojurians.slack.com/archives/C0G922PCH/p1574275574049800#2019-11-2111:04kszaboAnother way to solve is to post/prewalk the index and remove every key that has a function as a value#2019-11-2111:04kszabobut I did it the transit way as well#2019-11-2110:25miikkaHm, I've certainly used aleph+muuntaja+pathom, but not with the index explorer.#2019-11-2213:02Bjƶrn Ebbinghaus@wilkerlucio Do you know why the parser sometimes needs exactly 3s + the usual resolve time when I do not return a queried attribute and that is therefore set to :com.wsscode.pathom.core/not-found ?#2019-11-2213:20Bjƶrn EbbinghausThis happens every two to four queries and I don't know why.#2019-11-2216:21Spencer AppleI want to send a broadcast to via websocket with the updated information during a mutation. Where do you think in the pathom / fulcro stack the best place to do that is? I have no problem actually sending data via fulcro websocket’s push, I just am not sure the best place to call the push fn inside of pathom. 1. My first thought was just using the :parser in the env of a mutation to kick off an additional query and keep all the logic in the mutation. That didn’t work though as all my queries using the :parser in the env gave me a :com.wsscode.pathom.core/reader-error (which I don’t know how to debug). 2. Then I was thinking I could add a ::pc/transform function to mutations I want to broadcast, but I still am not sure which parser I can use to do that. 3. Also I could go the route of adding a tx-report-monitor or an atom watcher and broadcasting those changes on an async flow. Thanks for all your great work, and am curious of your thoughts#2019-11-2220:50Bjƶrn EbbinghausThe video is out: https://www.youtube.com/watch?v=IS3i3DTUnAI#2019-11-2315:01tianshuHi, I want to know you guys' tech stack in clojure/script for a fullstack development(using pathom). 😈#2019-11-2502:03wilkerluciomost of the times I write the servers from scratch, just pulling pathom, or depending on the kind of app, make it 100% client (Pathom can run there as well, in this model the client parser will process things and delegate to apis using the resolvers, using the async parser)#2019-12-0315:36Vincent Cantin@U0NBGRGD6 I am writing a template-based framework similar to VueJS where you can use fragments of EQL queries in place of data. It uses Pathom under the hood.#2019-12-0315:38tianshu@U8MJBRSR5 great! show me please!#2019-12-0315:39Vincent CantinI did not release its source code yet .. the poc was too primitive. I hope to have a first draft ready by the end of december.#2019-12-0315:40tianshudo you mean something like fulcro but without react?#2019-12-0315:41Vincent CantinSomething like Fulcro, but replace the render function with a template.#2019-12-0315:41Vincent Cantinby template I mean a declarative form .. a data.#2019-12-0315:43tianshuDo you have an API design at this moment?#2019-12-0315:43Vincent CantinHere is how it may look like:
(def todo-item
  {:id ::todo-item
   :name "Todo Item"
   :description "A todo item for the todo list."
   :props [^:eql 'item
           'color]
   :template `[:li
               (when color {:style {:color color}})
               (:todo-item/description item)
               (when (:todo-item/done? item)
                " (done)")]})

(def todo-list
  {:id ::todo-list
   :name "Todo List"
   :description "A todo list."
   :props [^:eql 'list]
   :env {'colors ["limeGreen" "chartreuse" "forestGreen"]
         'cycled-color (fn [colors index]
                         (get colors (mod index (count colors))))}
   :template `[:div
               [:h1 "Things to do"]
               [:div (:todo-list/title list)]
               (let [items (:todo-list/items list)]
                (if (empty? items) ; also can use if-not
                 [:div "No items in this list"]
                 [:ul
                  (for-indexed [[index item] items]
                   (let [item-color (cycled-color colors index)]
                    ^{:class :item} [::todo-item item item-color]))]))]})



(def my-components [todo-item todo-list])
#2019-12-0315:45Vincent CantinThat syntax will change, but that’s the idea#2019-12-0315:45tianshuinteresting! how you build the whole eql from root? I mean joined query, like [:x/a :a/b {:x/y [:y/c :y/d]}] .#2019-12-0315:46Vincent Cantin.. just follow the data usage.#2019-12-0315:47tianshuis that possible? I have been think for this for a while.#2019-12-0315:47Vincent Cantinme too, and now I am implementing it.#2019-12-0315:47tianshumay you share your idea?#2019-12-0315:48Vincent Cantinthe template is made of a DSL, not general Clojure functions.#2019-12-0315:48Vincent CantinI will talk more about it once I release the first draft.#2019-12-0315:48Vincent CantinThe name is Vrac.#2019-12-0315:50tianshuI see, use env to extract arbitrary clojure expression out of the template.#2019-11-2315:43tianshuI want to try a new stack (for me) which use eql and get the query from UI, resolve the query from database. So is fulcro/pathom/walkable a good choice?#2019-11-2501:58wilkerluciohello man, sorry the delay, still doing the trip back home :) #2019-11-2501:59wilkerlucioso, about walkable, one issue is that currently it doesnt integrate with connect (walkable was developed before that), so if you pull walkable you cant use the attribute connection thing on top of that#2019-11-2502:01wilkerlucioso, I suggest you can go without walkable, you can still implement the resolvers and run SQL queries (or anything else) in their body, not so automatic, but over time you really don't want to expose the whole SQL to the client#2019-11-2502:01wilkerluciomakes sense?#2019-11-2511:14tianshuaha, make sense.#2019-11-2511:17tianshuSo I can provide something like CRUD to the each resolver#2019-11-2512:40tianshuIs there a demo app for this? doesn't matter the database is datomic or sql.#2019-11-2522:01adamfeldmanIf you don’t want to write the CRUD SQL yourself, I’ve looked into using https://hasura.io/ to get GraphQL CRUD over Postgres, and then using Pathom Connect’s GraphQL support to interact with the GraphQL API generated by Hasura#2019-11-2522:02adamfeldmanIf you aren’t using Postgres, something similar may be possible with https://www.prisma.io/ (I am not sure, Prisma keeps pivoting their offerings…)#2019-11-2523:22Chris O’DonnellHere's a demo app using fulcro, pathom, and hasura: https://github.com/codonnell/crudless-todomvc#2019-11-2523:26adamfeldmanThanks @U0DUNNKT2!#2019-11-2523:27Chris O’DonnellSure thing šŸ™‚#2019-11-2622:57Ahmed HassanCrux seems promising too https://opencrux.com/#2019-11-2703:48tianshu@U0DUNNKT2 @UCHV4JZ7A @UCMNZLJ93 thanks for all you guys#2019-11-2522:01adamfeldmanIf you don’t want to write the CRUD SQL yourself, I’ve looked into using https://hasura.io/ to get GraphQL CRUD over Postgres, and then using Pathom Connect’s GraphQL support to interact with the GraphQL API generated by Hasura#2019-11-2517:58mattsfreyHey, I’m curious if anyone has tried using pathom in nodejs as a backend for fulcro and what the results were?#2019-11-2519:38Bjƶrn EbbinghausI got the weird situation in the Fulcro Inspect Query tab, I can't autocomplete mutations and always displays the following as the result
{:com.wsscode.pathom.core/reader-error
  "Mutation not found - {:mutation decide.model.account/update-display-name}"}}
But ... the mutation is there. Runs like expected and returns the value it is supposed to. It shows correctly in the network tab (Chrome and Fulcro Inspect). It shows up correct in the Index Explorer as well. Any ideas what is going on?
#2019-11-2811:08Bjƶrn Ebbinghaus@wilkerlucio I am trying to update the env in a mutation, so that I get the new db in the mutation-join. When I return just {::p/env env} I get a NullPointerException from the reader. Is there anything I have to consider where to update the env?#2019-11-2811:09Bjƶrn EbbinghausThe Exception:
#:decide.model.proposal{new-proposal #:com.wsscode.pathom.parser{:error #error {
 :cause nil
 :via
 [{:type java.lang.NullPointerException
   :message nil
   :at [com.wsscode.pathom.core$join invokeStatic "core.cljc" 423]}]
 :trace
 [[com.wsscode.pathom.core$join invokeStatic "core.cljc" 423]
  [com.wsscode.pathom.core$join invoke "core.cljc" 361]
  [com.wsscode.pathom.core$join invokeStatic "core.cljc" 370]
  [com.wsscode.pathom.core$join invoke "core.cljc" 361]
  [com.wsscode.pathom.connect$mutate_async$fn__28020$fn__28104$state_machine__8372__auto____28135$fn__28138 invoke "connect.cljc" 1516]
  [com.wsscode.pathom.connect$mutate_async$fn__28020$fn__28104$state_machine__8372__auto____28135 invoke "connect.cljc" 1511]
  [clojure.core.async.impl.ioc_macros$run_state_machine invokeStatic "ioc_macros.clj" 973]
  [clojure.core.async.impl.ioc_macros$run_state_machine invoke "ioc_macros.clj" 972]
  [clojure.core.async.impl.ioc_macros$run_state_machine_wrapped invokeStatic "ioc_macros.clj" 977]
  [clojure.core.async.impl.ioc_macros$run_state_machine_wrapped invoke "ioc_macros.clj" 975]
  [com.wsscode.pathom.connect$mutate_async$fn__28020$fn__28104 invoke "connect.cljc" 1511]
  [clojure.lang.AFn run "AFn.java" 22]
  [java.util.concurrent.ThreadPoolExecutor runWorker "ThreadPoolExecutor.java" 1128]
  [java.util.concurrent.ThreadPoolExecutor$Worker run "ThreadPoolExecutor.java" 628]
  [clojure.core.async.impl.concurrent$counted_thread_factory$reify__3036$fn__3037 invoke "concurrent.clj" 29]
  [clojure.lang.AFn run "AFn.java" 22]
  [java.lang.Thread run "Thread.java" 834]]}}}
#2019-11-2812:15wilkerlucio@U4VT24ZM3 thanks for the report, I think its a bug, I never used that in a mutation context#2019-11-2812:21wilkerluciochecking on it#2019-11-2812:28Bjƶrn EbbinghausGlad I could help. And good to know that I don't have to search any further for what I have done wrong.#2019-11-2812:30wilkerluciojust found it šŸ™‚#2019-11-2812:31wilkerluciojust gonna write some tests to cover it and send the fix soon#2019-11-2812:33Bjƶrn EbbinghausNice! šŸ‘#2019-11-2812:38wilkerluciohttps://github.com/wilkerlucio/pathom/pull/131#2019-11-2812:41wilkerlucioare you using deps? if you are, can you try pulling from master and see if works for you?#2019-11-2812:45Bjƶrn EbbinghausWill do. mom#2019-11-2812:52Bjƶrn EbbinghausWorks. šŸ‘#2019-11-2813:16wilkerluciothanks, I'll cut a new release soon#2019-11-2813:17magraHi, is there an alias-resolver that switches the E and V in EAV? I want to declare that
person :person/notes note
is equivalent to or can be resolved via
note :note/person person
Is this even possible?
#2019-11-2813:21Bjƶrn Ebbinghaus@magra Is this a pathom question? In Datomic you can make reverse lookups. https://docs.datomic.com/cloud/query/query-pull.html#reverse-lookup#2019-11-2813:22magraO it works but it hits the database twice.#2019-11-2813:25magraI write lots of queries [:person/id {:person/notes [:note/id :note/person]] because they preload the normalised DB in the App with the 'backlinks'. At the moment my resolver queries the db for :note/_person, dissoces that and then assoces it as :person/notes. But half the time it hits the DB again to get :note/person becaue it does not 'see' the relationship between :note/person and :note/_person.#2019-11-2813:27magraMaybe I am thinking wrong. But the E in EAV is always implicit.#2019-11-2813:30wilkerlucio@magra I think to get efficient with that, the ideal would be to send the reverse direct to datomic as part of the pull request, had you tried https://github.com/wilkerlucio/pathom-datomic ? still very very alpha, but I think it supports that, have to try and check#2019-11-2813:32magraIt has been some time since I checked that. At the moment I do send it directly to the db. If that is the most efficient I will keep it that way. Thanks!!#2019-11-2813:32wilkerluciothat's more true for datomic cloud, on prem it makes little difference#2019-11-2813:32magraI will study pathom-datomic.#2019-11-2813:33wilkerlucioyeah, I'm excited for the new query planner I'm working on, it will be a great improve to the dynamic resolvers integration story (that includes Datomic, GraphQL, Pathom <> Pathom, SQL...)#2019-11-2813:36Mark AddlemanWhat's the Pathom story around queries with aggregations? I'm working on an analytics product where the user can create arbitrary queries so slice and dice data. We're suffering with my badly designed DSL to describe aggregate queries and I'd love to switch to something much better thought through and battle tested#2019-11-2813:38wilkerlucioPathom com.wsscode/pathom "2.2.27" was just released, this fixes a but when trying to augment env with connect mutations, also adds support for docstrings (the become data on the resolver map) on pc/defresolver#2019-11-2813:40wilkerluciohello @mark340, I personally don't have much experience doing it, but I would use EQL parameters to do it, they are an open dimension of information you can use to describe anything you want, you can take Walkable interface as inspiration: https://walkable.gitlab.io/aggregators.html#2019-11-2813:41wilkerlucioone suggestion I have if you go in that direction, use namespaced keywords in your interface definition (the names of the parameters), this way keep it open to integrate with other possible definitions you may want to use in the future#2019-11-2813:42Mark AddlemanCool. I'll check this out. Thanks!#2019-11-2813:45Mark AddlemanOne more thing: For the most part, our schema is aribitrary key-value pairs. In order to support this in Pathom, I guess I'd write my own resolver that knew how to convert an EDN key-value pair into the appropriate SQL addressing?#2019-11-2813:47wilkerluciothat's some room for options in this space, really depends how you wanna go about modeling it#2019-11-2813:47wilkerlucioso just to see if we are in the same page, your target database is some SQL, is that correct?#2019-11-2813:50Mark AddlemanYep. More specifically, our tables have arbitrarily named columns. It's not too weird from the database perspective. It's just that the UI queries are metadata driven#2019-11-2813:51wilkerluciogiven this, you want a solution that automatically makes all the table options available, or something more controled on the API side?#2019-11-2813:53Mark AddlemanWhat does "table options" mean? Generally, we're trying to provide a high degree of flexibility to the API#2019-11-2813:55wilkerlucioI mean, you can try to do things like "introspect my table schema, and generate the resolvers for all of it", or more like: "ok, I'm going to give the user this specific list with these specific fields, and via params I'll customize just filter/aggregation on top of this specific thing"#2019-11-2813:55wilkerluciomakes sense?#2019-11-2813:55Mark AddlemanYes. Much more like introspection.#2019-11-2813:55Mark AddlemanBut perhaps the implementation matters#2019-11-2813:56Mark AddlemanWe have a metadata that describes all the columns available to the user#2019-11-2813:56Mark Addleman*metadata table#2019-11-2813:56wilkerlucioyup, the instropection version is much harder to get right#2019-11-2813:57Mark AddlemanUnlike naive introspection using the db's information schemas, we have a very rich understanding of the columns, their types, etc#2019-11-2813:57wilkerlucioI guess it may be easier if start talking more concrete, hehe, what kind of querys you would like to support in the system?#2019-11-2813:58wilkerlucioor putting more simply, what are the user inputs?#2019-11-2813:59Mark AddlemanThe user inputs are group-by columns, aggregate columns and functions, and a where clause. the one complication is that the where clause can reference another table that must be joined in#2019-11-2814:00Mark AddlemanOh and time range#2019-11-2814:01wilkerluciook, so its some sort of graphic SQL interface#2019-11-2814:01Mark Addlemanexactly#2019-11-2814:01Mark AddlemanThe generated SQL can be complicated due to UI niceties. For example, the user may ask for some aggregation grouped by day of week. If the data happens not to have Sunday, for example, we still want the#2019-11-2814:02Mark Addlemanresult set to include Sunday with some default value (null or zero)#2019-11-2814:02wilkerlucioseems like Walkable is a good option for you, currently the bad part about it is that it currently doesn't integrate with Connect, so you don't get the graph traversal and auto-complete things
#2019-11-2814:03wilkerluciobut for a lot of the things you are describing, he seems to have already, aggregations, joins, etc...#2019-11-2814:03wilkerlucioso you can convert your user input directly into some EQL representation supported by Walkable, and let it run it#2019-11-2814:03Mark AddlemanOk. Auto complete isn't a requirement - at least, that is being handled by a separate system#2019-11-2814:03Mark AddlemanThat sounds promising. I'll investigate.#2019-11-2814:03Mark AddlemanThanks a ton#2019-11-3013:29tianshucan I just re-run defresolver instead of redefine the parser to make my change available?#2019-12-0100:33wilkerluciono, if you run defresolver it will just redefine the map, but the index is built on parser construction, so that has to be reloaded as well#2019-12-0215:04tianshuthanks! I found I can reload the whole changes by something like cider-ns-refresh#2019-12-0220:11Bjƶrn Ebbinghaus@wilkerlucio Does pathom support bounded recursion? Like in the eql specs? https://edn-query-language.org/eql/1.0.0/specification.html#_recursive_queries#2019-12-0220:11wilkerlucioyup#2019-12-0220:12wilkerlucioits supposed to, but to be fair I don't even remember last time I tried it, so may be buggy, if you find some issue with it please let me know#2019-12-0220:12Bjƶrn EbbinghausDo I have to do anything special? It seems like the recursion depth int is "leaking" into my resolvers. class java.lang.IllegalArgumentException: Don't know how to create ISeq from: java.lang.Long#2019-12-0220:13Bjƶrn EbbinghausActually forget the "leaking". The exception has to come from pathom. not my resolver.#2019-12-0220:14wilkerluciocan you get me a small reproduction example of the issue?#2019-12-0220:18wilkerluciono, it should work the same as ...#2019-12-0220:18wilkerlucio(but with the bound limit, of couse :))#2019-12-0220:22Bjƶrn EbbinghausI investigated.. When I disable the error-handler-plugin I see that the Exception is thrown by transit on the way out..#2019-12-0220:25Bjƶrn Ebbinghaus
[{:type java.lang.IllegalArgumentException
   :message "Don't know how to create ISeq from: java.lang.Long"
   :at [clojure.lang.RT seqFrom "RT.java" 557]}]
 :trace
 [[clojure.lang.RT seqFrom "RT.java" 557]
  [clojure.lang.RT seq "RT.java" 537]
  [clojure.core$seq__5402 invokeStatic "core.clj" 137]
  [clojure.core.protocols$seq_reduce invokeStatic "protocols.clj" 24]
  [clojure.core.protocols$fn__8136 invokeStatic "protocols.clj" 75]
  [clojure.core.protocols$fn__8136 invoke "protocols.clj" 75]
  [clojure.core.protocols$fn__8088$G__8083__8101 invoke "protocols.clj" 13]
  [clojure.core$transduce invokeStatic "core.clj" 6884]
  [clojure.core$into invokeStatic "core.clj" 6899]
  [clojure.core$into invoke "core.clj" 6887]
  [edn_query_language.core$query__GT_ast invokeStatic "core.cljc" 302]
  [edn_query_language.core$query__GT_ast invoke "core.cljc" 295]
  [com.wsscode.pathom.core$join_seq_parallel$fn__18611$state_machine__8483__auto____18624$fn__18627 invoke "core.cljc" 436]
  [com.wsscode.pathom.core$join_seq_parallel$fn__18611$state_machine__8483__auto____18624 invoke "core.cljc" 432]
  [clojure.core.async.impl.ioc_macros$run_state_machine invokeStatic "ioc_macros.clj" 973]
  [clojure.core.async.impl.ioc_macros$run_state_machine invoke "ioc_macros.clj" 972]
  [clojure.core.async.impl.ioc_macros$run_state_machine_wrapped invokeStatic "ioc_macros.clj" 977]
  [clojure.core.async.impl.ioc_macros$run_state_machine_wrapped invoke "ioc_macros.clj" 975]
  [com.wsscode.pathom.core$join_seq_parallel$fn__18611 invoke "core.cljc" 432]
  [clojure.lang.AFn run "AFn.java" 22]
  [java.util.concurrent.ThreadPoolExecutor runWorker "ThreadPoolExecutor.java" 1128]
  [java.util.concurrent.ThreadPoolExecutor$Worker run "ThreadPoolExecutor.java" 628]
  [clojure.core.async.impl.concurrent$counted_t
#2019-12-0220:28Bjƶrn EbbinghausOk. Transit is trying to encode the Exception. I was confused.#2019-12-0220:37Bjƶrn EbbinghausI suspect that I'm the murderer...#2019-12-0220:38Bjƶrn EbbinghausOr am I?#2019-12-0220:47wilkerluciosorry, just looking at the stack I'm not sure whats going on, but if you can give me a small reproduction case I can debug it out#2019-12-0220:48wilkerlucioit works if you use unbounded recursion?#2019-12-0221:17Bjƶrn EbbinghausYes. Sorry, that I did not reply. I have some problems getting the simplest resolver to run in the repl..#2019-12-0221:52Bjƶrn EbbinghausI forgot the reader... šŸ˜“ My example works like intended...#2019-12-0222:08Bjƶrn EbbinghausAh! Got it! @wilkerlucio The Bug only appears with the parallel-parser. The serial parser works fine.#2019-12-0222:11Bjƶrn Ebbinghaushttps://gist.github.com/MrEbbinghaus/88d84683a5be5908b854cdfc081fc5eb#2019-12-0222:14Bjƶrn EbbinghausAnd it also only happens when I do a to-many join#2019-12-0313:08wilkerluciocool, thanks! I'll take a look later today#2019-12-0412:58Bjƶrn EbbinghausI opened an issue so that this is not being swallowed by Slack. https://github.com/wilkerlucio/pathom/issues/132#2019-12-0300:41roklenarcicif there a branch or something of pathom where graphql fulcro remote works with Fulcro 3?#2019-12-0300:41roklenarcicseems to be using Fulcro 2 namespaces currently#2019-12-0310:58souenzzo@roklenarcic should be easy to implement a fulcro-remote in fulcro3. "just a function"#2019-12-0317:03roklenarcicwell I can implement anything, the issue is that I told other people that fulcro 3 had graphql integration through pathom, which turned out to be untrue#2019-12-0317:38wilkerlucio@roklenarcic its not untrue, this didn;t really change between 2 and 3, the only difference is that there is no pre-made Pathom parser to hook into the Fulcro network on f3, but that's something quite trivial to implement#2019-12-0416:09Spencer AppleIs there a way for a resolver to know if a mutation is joined with it?#2019-12-0416:09Spencer Appleie. a mutation changed state and returned an id, which then a resolver joined to and resolved the entity#2019-12-0416:10Spencer Applei'd like to trigger a websocket push on the resolver only if a mutation triggered the resolver#2019-12-0416:15eoliphantI dont remember the specifics but I "believe" there's info in the env that might help you. I was doing something with plugins some time ago where that came up. But @wilkerlucio will have the definitive answe#2019-12-0502:01wilkerlucioHello everybody, I just released pathom 2.2.28, in this version: • fix bounded recursive queries on parallel parser @mroerni • connect mutations add the key ::pc/mutation-ast to env , @splayemu you can use this now to see the source mutation when doing follow up reads#2019-12-0502:02Spencer Appleah great, thanks for your work!#2019-12-0510:39souenzzoMutation ast will be awesome for ingĆ” :)#2019-12-0513:54fjolneHi there! I’m trying to get a root resolver for ::current-user, which is resolved from the ::current-session, which doesn’t require any inputs. So, instead of having to write [{::current-session [{:session/user [:user/id]}]}] every time, I want to be able to get the same result by [::current-user]. Seems like it would require to be able to resolve attributes inside of the resolver itself, something like
(defresolver current-user-resolver [_ {::keys [current-session]}]
  {::pc/input #{{::current-session [{:session/user [:user/id]}]}}
   ::pc/output [{::current-user [:user/id]}]}
  {::current-user (select-keys (:session/user current-session) [:user/id])})

(defresolver session-resolver [{:keys [db]} {:session/keys [id]}]
  {::pc/input  #{:session/id}
   ::pc/output [{:session/user [:user/id]}]}
  (d/pull db [{:session/user [:user/id]}] [:session/id id]))

(defresolver current-session-resolver [{{:keys [session]} :ring/request} _]
  {::pc/output [{::current-session [:session/id]}]}
  {::current-session (select-keys session [:session/id])})
Is there a way to achieve that without writing a custom DB query (given that Pathom Connect knows how to get the needed data)?
#2019-12-0912:44wilkerlucioI guess what you are looking for is nested inputs, this is something I like to support, but not there yet, there is a way around it now, you can do a manual call to the parser with the input sub-query, something like this:
(defresolver current-user-resolver [{:keys [parser] :as env} {::keys [current-session]}]
  {::pc/input  #{::current-session}
   ::pc/output [{::current-user [:user/id]}]}
  ; note: if you are using async or parallel parsers, the parser return is a channel and
  ; you have to add code to wait for it
  (let [current-session (-> (parser env [{::current-session [{:session/user [:user/id]}]}])
                            ::current-session)]
   {::current-user (select-keys (:session/user current-session) [:user/id])}))

(defresolver session-resolver [{:keys [db]} {:session/keys [id]}]
  {::pc/input  #{:session/id}
   ::pc/output [{:session/user [:user/id]}]}
  (d/pull db [{:session/user [:user/id]}] [:session/id id]))

(defresolver current-session-resolver [{{:keys [session]} :ring/request} _]
  {::pc/output [{::current-session [:session/id]}]}
  {::current-session (select-keys session [:session/id])})
#2019-12-0914:11wilkerluciomakes sense?#2019-12-1005:17fjolneYes, thank you! That’s what I was looking for (and also for the approval that I’m not misunderstanding some principles).#2019-12-0915:03mitchelkuijpersIs it possible with pathom to add something like: :roles-needed [:admin] to the defresolver map?#2019-12-0915:03mitchelkuijpersAnd then use that in a plugin to check roles?#2019-12-0915:03mitchelkuijpersI have it working a bit by checking: (::pc/resolver-data env) that would then include :roles-needed#2019-12-0915:05mitchelkuijpersI have something like [{:foo [:secret]}] if have a resolved for this I can find :roles needed when querying for this. But when I just query for :foo it will simply return the value without me knowing that there are roles needed#2019-12-0915:57Bjƶrn Ebbinghaus@mitchelkuijpers Yes. It is. I have something similar, where you can get inspiration. I wrote a plugin so that I can give my mutations a spec for their parameters.
clojure
(def spec-plugin
  {::p/wrap-mutate
   (fn [mutate]
     (fn [env sym params]
       (if-let [spec (get-in env [::pc/indexes ::pc/index-mutations sym ::s/params])]
         (if (s/valid? spec params)
           (mutate env sym params)
           (do
             (log/warn (s/explain spec params))
             (throw (ex-info "Failed validation!" (s/explain-data spec params)))))
         (mutate env sym params))))})

(defmutation set-vote [{:keys [connection AUTH/account-id] :as env} {proposal-id :proposal/id
                                                                     account     :account/id :as params}]
  {::pc/params [:proposal/id :account/id :account/id]
   ::pc/output [:proposal/id]
   ::s/params  ::upsert-vote}
 ....)
#2019-12-0916:00Bjƶrn Ebbinghaus@wilkerlucio While I am at it. Is there a better way to get the current mutation/resolver map in a plugin other than (get-in env [::pc/indexes ::pc/index-mutations sym ::s/params]) ?#2019-12-0918:59mitchelkuijpersAh that helps a lot thank you @mroerni#2019-12-0919:42wilkerlucio@mroerni that's the intended way, I plan to keep those keys stable, so it shoulnd't break, but I can add a few helpers to facilitate the access, something like (pc/mutation-data env 'my-mutation), what you think?#2019-12-0920:05mitchelkuijpers@wilkerlucio for reads do I first need to get the sym from [::pc/indexes ::pc/index-attributes] and then to :attr-output-in and then get the symbol from [::pc/indexes ::pc/indexes-resolvers] ?#2019-12-0920:12mitchelkuijpers
(defn get-resolver [env]
  (let [k (get-in env [:ast :key])
        sym (get-in env [::pc/indexes ::pc/index-attributes k ::pc/attr-output-in])]
    (reduce
     (fn [m k]
       (merge m (get-in env [::pc/indexes ::pc/index-resolvers k])))
     {}
     sym)))
This seems to work, very nifty those maps with all the paths
#2019-12-1110:20thosmosI’m attempting to make a single-input resolver to handle a pull-like query based on an ident like this: [{[:agency/id "123"] [:db/id :agency/Active :agency/Name ,,, {:agency/Projects [:db/id :project/Name ,,,]}]}] I’m needing the query and the list of keys, but the problem is that the :query key is missing from the AST and only the first query key is mentioned in the AST: {:type :prop, :dispatch-key :db/id, :key :db/id} What’s the right way to do this? The resolver returns everything if I just blindly do a [*] wildcard pull, but I’d rather shape it for the actual query#2019-12-1110:42wilkerlucio@thosmos resolvers are triggered during attribute processing, that's the reason you get the single :db/id, to look back you can use ::p/parent-query from env, I guess that's the one you are looking for#2019-12-1110:44wilkerlucio@mitchelkuijpers yup, you can use those like that šŸ‘#2019-12-1118:51thosmos@wilkerlucio thanks I missed that when I looked at the env keys!#2019-12-1119:10thosmosWhat’s doing the sub-select of the AST before handing it to the resolver? I ask because to me these two queries are effectively the same context but the second just narrows the scope down to one result: [{:agencies [:db/id ,,,]}] [{[:agency/id 1][:db/id ,,,]}] In other words, how I think about it, :agencies and [:agency/id 1] are the root keys in these two similar queries, and as I’m attempting to resolve these, I would expect the :agency/id to be in the the root node of the AST with all of its first level query attributes as its children. In order to get this, would I modify the reader?#2019-12-1213:14wilkerlucio@thosmos the :agency/id goes in the entity, which is a value in the env (an atom so it can be modified in a sideway and concurrently), the first one you send its an open link, can be one or many, but idents are always going to be a single entity, but I think you are complecting the data and the AST#2019-12-1213:14wilkerluciothe ident-reader moves the thing from the ident joins to context data, while the :agencies are your own resolver#2019-12-1213:15wilkerluciobut you should get the data using resolver inputs, are you having any trouble to deal with them this way?#2019-12-1216:31thosmosI’m not having any trouble. I got it working now by using the env’s ::p/parent-query key as you suggested. But I realize I have a preference. My familiarity is from parsing EQL on the server side via Om/Next where the AST had everything I needed to process the request because it had info about the root key and the query keys, which included the ident key as the parent or root key, and the query keys as children. Currently, there’s insufficient information in the AST to process the request, since the AST only provides the first key from the query. I like that the ident is in the input of the resolver, but not that it’s missing from the AST. But, I don’t want to take any more or your time. I’ll just look at the code; if I want it to behave differently, I’ll figure out how to make it.#2019-12-1217:18wilkerlucioyeah, Pathom really tries to abstract a lot of that away from you, manipulations with AST should be used only for extensions and special cases, for regular things you shouldn't need it at all, that's the intended interface#2019-12-1219:51thosmosok got it#2019-12-1503:34Alister LeeOla! What is the suggested way for a resolver to access the session? (scenario is that resolver returns different result depending on the user.) I feel like the env is established with the middleware so doesn't have access to the request.#2019-12-1608:17Alister LeeOk I think I get it - a fulcro server problem really - not possible with wrap-api, but should be easy to inject.#2019-12-1610:50wilkerlucio@clojure388 hello, one thing also to remember, the first argument you send to the parser call is also environment, so you can add anything you want there#2019-12-1610:52Alister LeeGot it, thanks - you want the session and the database connection coming in through the environment - the pathom docs even show this iirc. I need to wrangle fulcro/middleware to get it there.#2019-12-1623:46mdhaneyIn one of Tony’s videos, he shows how to write a replacement for wrap-api to do exactly what you are asking for. It’s this one IIRC: https://youtu.be/oQpmKWBm9HE#2019-12-1701:47Danny AlmeidaI'm trying pathom documentation ...https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/resolvers.html#_parameters#2019-12-1701:48Danny Almeidai've tried the example..but when i enter this query [(::instruments {:sort :instrument/brand})]in the REPL it gives me an error#2019-12-1701:49Danny Almeidaany idea what I might be doing wrong#2019-12-1702:05wilkerluciohello @dionysius.almeida, welcome šŸ™‚, strange, I just tried that and it worked here, are you getting it consistently?#2019-12-1702:41Danny Almeidayes...#2019-12-1702:41Danny AlmeidaI'm trying it in a clojure repl#2019-12-1702:41Danny Almeidanot on the web page..where it works fine#2019-12-1702:44Danny Almeida(def instruments [{:instrument/id 2, :instrument/brand "Tajima", :instrument/type :instrument.type/ukulele, :instrument/price 50} {:instrument/id 4, :instrument/brand "Cassio", :instrument/type :instrument.type/piano, :instrument/price 160} {:instrument/id 3, :instrument/brand "Ibanez", :instrument/type :instrument.type/bass, :instrument/price 270} {:instrument/id 1, :instrument/brand "Fender", :instrument/type :instrument.type/guitar, :instrument/price 300}]) (pc/defresolver instruments-list [env _] {::pc/output [{::instrumentlist [:instrument/id :instrument/brand :instrument/type :instrument/price]}]} (let [{:keys [sort]} (-> env :ast :params)] {::instrumentlist (cond->> instruments (keyword? sort) (sort-by sort))}))#2019-12-1702:46Danny Almeidaand followed by (<!! (parser {} [(::instrumentlist {:sort :instrument/brand})]#2019-12-1702:48Danny Almeida#error { :cause "Invalid expression " :data {:type :error/invalid-expression} :via [{:type clojure.lang.ExceptionInfo :message "Invalid expression " :data {:type :error/invalid-expression} :at [com.wsscode.pathom.parser$expr__GT_ast invokeStatic "parser.cljc" 100]}] ~#2019-12-1702:48Danny AlmeidaThat's the error message I get#2019-12-1705:14thosmosIs it possible to send metadata across the wire? OR how do you handle sending the result count of a query (before limit and offset)? Do you do a second meta query with no keys and a limit of 0 just to get the total count?#2019-12-1705:40wilkerlucio@dionysius.almeida the issue is that you not escaping the call to the parser, so its invoking the keyword on that map, you need to use: (<!! (parser {} ['(::instrumentlist {:sort :instrument/brand})]#2019-12-1705:41wilkerlucio@thosmos yes, you just need to setup transit so it send it, I have a use case for that were we track which components are spending time of the query fragments#2019-12-1705:41Danny Almeidaoh i see...thank you. I'm still new to pathom..just started reading the docs today as part of learning Fulcro šŸ™‚#2019-12-1705:42thosmos@wilkerlucio is that related to the trace option I see in fulcro 3?#2019-12-1705:43wilkerluciono, I don't think so#2019-12-1705:43wilkerluciothe question on metadata over the wire is purely a transit thing#2019-12-1705:43wilkerluciowhich trace option are you talking about?#2019-12-1705:44thosmosoh there’s something in fulcro inspect that can be turned on by running the build with a -Dtrace option and it adds some stuff to the queries#2019-12-1705:44thosmosis there a code example of the metadata on client/server sides that I can see?#2019-12-1705:45thosmosI think I see where to config transit on the server side, currently looking at the Fulcro3 client side aspect#2019-12-1705:46Danny Almeida@wilkerlucio That worked! Thanks again šŸ‘:skin-tone-3:#2019-12-1705:47thosmosok, I think I see where to make the changes. thanks!#2019-12-1816:37colinkahnIs it advisable to call my parser within a mutation, for example if I want to call other mutations I've registered?#2019-12-1820:58wilkerlucioI suggest instead you make the mutations more atomic like. they are a final entry point, its better if you abstract the internals so you can share that between mutations, makes sense?#2019-12-2322:11colinkahn@U066U8JQJ yep, definitely. Is it common to do other parser queries within mutations? Like lets say I need to know the current state of something. Or should the parser be considered more of an interface for the outside world and internally use whatever storage i’m building against to query for things?#2019-12-2322:34wilkerlucio@U0CLLU3QT you usually don't need it, the response of a mutation usually has some data about what was transacted, if the user do a mutation join (running a query against the result), them pathom will already trigger the resolver engine to consolidate the gap between the mutation output and the user data, this is what I see happening most of the time#2019-12-2322:37colinkahn@U066U8JQJ thanks for the insight, really liking pathom so far!#2019-12-1817:15kszaboyes, that’s idiomatic#2019-12-1820:52souenzzo
(let [register [(pc/resolver `foo
                             {::pc/output [::foo]}
                             (fn [{::keys [foo]} _]
                               {::foo foo}))
                (pc/resolver `parser
                             {::pc/output [::parser]}
                             (fn [{:keys [parser] :as env} _]
                               {::parser [(parser (assoc env ::foo 1)
                                                  [::foo])
                                          ;; how do I "cleanup" env cache here?
                                          (parser (assoc env ::foo 2)
                                                  [::foo])]}))]
      p (p/parser
          {::p/env     {::p/reader               [p/map-reader
                                                  pc/reader2
                                                  pc/open-ident-reader
                                                  p/env-placeholder-reader]
                        ::p/placeholder-prefixes #{">"}}
           ::p/plugins [(pc/connect-plugin {::pc/register register})]})]
  (p {} [::parser]))
How do I control/cleanup env cache?
#2019-12-1821:27souenzzousing
(assoc env
  ::p/request-cache (atom {})
  ::p/entity (atom {}))
#2019-12-2016:07Vincent Cantinhello, I did not fully understand the syntax for recursive queries. I saw to-where the queries should recurse via "...", but I did not see the from-where.#2019-12-2016:08Vincent Cantindoes it mean that the from-where is always the root of the query ?#2019-12-2016:16wilkerlucio@vincent.cantin the recursion is always on the same level (parent), like datomic pull syntax, makes sense?#2019-12-2016:53Vincent Cantinyes, thank you#2019-12-2206:11Vincent CantinI wonder if EQL could be extended to allow general recursion, by specifying the ā€œfromā€ and ā€œtoā€ recursion points#2019-12-2206:13Vincent Cantinsomething that could work with recursions like [{person [{group [{person [{group …#2019-12-2207:33Vincent Cantinmaybe via meta ?#2019-12-2216:58wilkerlucio@vincent.cantin do you have a real use case that seems to require modeling like this? I wonder what that looks like#2019-12-2301:40Vincent CantinMy current use case is the implementation of Vrac itself: any component can refer to any component, and cross pathom-context recursion may arise as a result of the user’s components composition. I guess that Fulcro had to face the same issue with the get-query function at some point. I will take a look at how it is implemented.#2020-12-3019:49henrikFor a stack that uses Pathom, how do you normally deal with APIs for non-Clojure clients, such as a mobile apps?#2020-12-3019:54eoliphantI’ve been messing around with mapping an RPC-style API onto existing resolvers#2020-01-0115:18dnsHi! I've probably missed something in the documentation but I can't figure this out. Any help is appreciated! Suppose I have two resolvers as in:
; resolver 1
::pc/input #{:x/id}
::pc/output [:x/name {:x/y [:y/id]}]

; resolver 2
::pc/input #{:y/id}
::pc/output [:y/name]
How can I write a resolver that, given :x/id, can calculate some function based on :x/name and :y/name? I've tried the following:
::pc/input #{:x/id :x/name :y/name} ; always returns not-found
::pc/input #{:x/id :x/name :x/y} ; won't have :y/name
::pc/input #{:x/id :x/name {:x/y [:y/name]}} ; won't work
#2020-01-0121:37fjolne@dns I’ve had essentially the same question https://clojurians.slack.com/archives/C87NB2CFN/p1575895495389300?thread_ts=1575554067.386000&amp;channel=C87NB2CFN&amp;message_ts=1575895495.389300#2020-01-0915:23fjolneHi there! I’m wondering what is the recommended approach for conditional dependencies on attributes. Idiomatic approach seems to be not returning the attribute if it’s absent:
(defresolver test-x [_ _]
  {::pc/input  #{}
   ::pc/output [:x]}
  {})

(defresolver test-y [_ _]
  {::pc/input  #{:x}
   ::pc/output [:y]}
  {:y true})
But that would throw an error when trying to access :y (which might be fine to manually elide, but then we need to distinguish real errors from such ones). The other approach would be to return some absent value (e.g. nil), but that doesn’t seem idiomatic and requires more handling for each case:
(defresolver test-x [_ _]
  {::pc/input  #{}
   ::pc/output [:x]}
  {:x nil})

(defresolver test-y [_ {:keys [x]}]
  {::pc/input  #{:x}
   ::pc/output [:y]}
  {:y (boolean x)})
#2020-01-0916:11wilkerlucio@fjolne.yngling pathom assumes the following: • if an attribute is not returned, this means "I don't know to get that here, but other resolver may be able to", so other resolvers will be tried if that attribute is required • if you return nil, them its assume nil is the final value, so other resolvers will not be attempted#2020-01-0916:12wilkerluciothe situation you talk, seems more a problem of optional input than output, because I guess you wanna run test-y even if x is not available, is this correct?#2020-01-0916:17fjolne@wilkerlucio thanks for the quick response, I understand what you’re saying, but I have the opposite case: I don’t want to run test-y if there’s no :x. For now I’m using elide-special-outputs-plugin to sweep ::p/reader-error, but it seems more appropriate if reader would return ::p/not-found if it couldn’t resolve transitive deps for :y in my case.#2020-01-0916:18wilkerluciooh, it should be not-found in this case, you are seeing :x with reader-error?#2020-01-0916:19fjolneI’m seeing reader-error for [:y] tx, and not-found for [:x] tx#2020-01-0916:23wilkerluciook, that sounds more like a bug, :y should be not found#2020-01-0916:25fjolneI’ve just checked that it’s actually not-found for parallel-reader, but error for reader2#2020-01-0916:26fjolneand I switched to reader2 because parallel-reader accidentally hang forever in some weird cases#2020-01-0916:31wilkerluciosorry the mess, can you open an issue for it? I can check it out at some point this week#2020-01-0916:40fjolne@wilkerlucio sure, np https://github.com/wilkerlucio/pathom/issues/137#2020-01-0916:45souenzzoEasy single sexp reproduce
(let [register [(pc/resolver `x1
                             {::pc/output [:x1]}
                             (fn [env input] {:x1 nil}))
                (pc/resolver `x2
                             {::pc/output [:x2]}
                             (fn [env input] {}))
                (pc/resolver `y1
                             {::pc/input  #{:x1}
                              ::pc/output [:y1]}
                             (fn [env {:keys [x1]}] {:y1 (pr-str x1)}))
                (pc/resolver `y2
                             {::pc/input  #{:x2}
                              ::pc/output [:y2]}
                             (fn [env {:keys [x2]}] {:y2 (pr-str x2)}))]
      parser (p/parallel-parser {::p/plugins [(pc/connect-plugin {::pc/register register})
                                              p/error-handler-plugin]})
      env {::p/reader [p/map-reader
                       pc/reader2
                       pc/open-ident-reader]}]
  (async/<!! (parser env [:y1 :y2])))
#2020-01-0916:47souenzzoOutput with pathom 2.2.8
{:y1 "nil",
 :y2 :com.wsscode.pathom.core/reader-error,
 :com.wsscode.pathom.core/errors {[:y2] "class clojure.lang.ExceptionInfo: Insufficient resolver output - {:com.wsscode.pathom.parser/response-value {}, :key :x2}"}}
#2020-01-0917:03fjolne@U2J4FRT2T thanks, added to the issue#2020-01-0917:34Bjƶrn Ebbinghaus@wilkerlucio Are you aware of this issue? https://github.com/wilkerlucio/pathom/issues/135#2020-01-0917:38wilkerlucioI see @U2J4FRT2T just replied on that, can you confirm its the same thing for you?#2020-01-0917:43souenzzoIt's a printing loop issue, like (let [a (atom nil)] (reset! a a))#2020-01-0918:06Bjƶrn EbbinghausWhat does this have to do with printing?#2020-01-0918:09souenzzoit has (many) recursive references (due atoms) When clojure tryies to print it, it thows a stackoverflow error. but when you do things like (keys (parser ...)) , it will not try to print all the structure, just the keys. So, pathom is returning as expected.#2020-01-0918:10souenzzomaybe we can add a behavior to pathom always dissoc ::p/env key from map before return. It can "fix" this issue, but I don't think that we want this behavior#2020-01-0918:19Bjƶrn EbbinghausI understand that printing this recursion leads to the StackOverflow in this case. But why should this recursive reference even be in the result? Mutating without a query is fine. And updating the env in a mutation is fine as well. So pathom should not fail in this situation. Or do I misunderstand something here?#2020-01-0918:20Bjƶrn EbbinghausMaybe "fail" is the wrong word here.#2020-01-0918:22Bjƶrn EbbinghausThese two queries should both have the same result, but they don't
(my-mutation {})
{(my-mutation {}) []}

=> {:my-mutation {}}
#2020-01-0918:29wilkerluciothanks for the clarification @U4VT24ZM3, I believe up to Friday I'll have some spare time, so I can check on all these issues in one go, thanks for the pacience on this one#2020-01-0918:30Bjƶrn EbbinghausNo hurry. The issue is open for a month and the workaround is trivial. I forgot about the issue myself and the comment from @U2J4FRT2T reminded me.#2020-01-0919:33Ben GrabowI've skimmed through the Pathom docs but I haven't noticed any mention of subscriptions. Is there any support for server-push subscriptions in pathom or is it just pull-based queries? https://graphql.org/blog/subscriptions-in-graphql-and-relay/#2020-01-0919:43kszabono support yet#2020-01-0919:44kszabohttps://github.com/wilkerlucio/pathom/issues/98#2020-01-0919:45kszabois the closest issue we had around this, at least in the same space of a persistent connection that emits new data#2020-01-0920:15Ben GrabowWould it help if I type up some details of my use case for subscriptions and add them to that issue? Push-based updates are a big part of my company's use case so I'd love to see a push-based EQL solution in the future.#2020-01-0920:27kszaboI think that would be helpful šŸ™‚#2020-01-0921:11Ben Grabowhttps://github.com/wilkerlucio/pathom/issues/98#issuecomment-572754191#2020-01-1213:30wilkerlucio[com.wsscode/pathom "2.2.29"] is out! a few fixed bugs: - pc/reader2 will return ::p/not-found instead of an error when dependency links are missing @fjolne.yngling - ::p/env is automatically removed from mutations that return it and have no query @mroerni Check the full changelog at: https://github.com/wilkerlucio/pathom/blob/master/CHANGELOG.md#2020-01-1213:31wilkerlucio@fjolne.yngling I decided to not fix the parallel-reader case at this time, the reason is that there are some complex usages of it, and chancing that may break some critical scenarios, so I rather keep it consistent there#2020-01-1220:04fjolne@wilkerlucio great, thanks! i’m not sure what you mean about parallel-reader though, it seems to be working as expected#2020-01-1220:04wilkerlucioI mean the second example you add after, reported from @U2J4FRT2T#2020-01-1220:06fjolnebut there’s no parallel-reader in that example? for me it’s also returning not-found#2020-01-1220:23wilkerluciohum, you right, I saw him using the parallel-parser and assumed he was using the parallel-reader, in this case it should be fixed as well, I'll double check#2020-01-1408:38levitanong@wilkerlucio not sure if i’m misunderstanding how resolvers work, but I created an ident resolver to get additional information over the network about a specific entity. Additional information was to be passed in params, so in my call to df/load, i passed a params map expecting that in my resolver I could get it from the AST. However, when I looked at the AST inside the resolver, it was a :prop AST for only one of several props i declared in my output. What i expected was an AST that was of type :root, and had a :params key in there. Am I misunderstanding the usage of this, or is this a bug?#2020-01-1410:10Bjƶrn EbbinghausCan you provide the query?#2020-01-1411:25levitanong
(defsc FooWithAdditionalInformation
  [this props]
  {:ident [:foo/by-id :foo/id]
   :query [:foo/id :foo/bar :foo/baz]})

(df/load this [:foo/by-id 0] FooWithAdditionalInformation
  {:params {:other-data other-data}})
Inside the resolver the ast is:
{:type :prop, :dispatch-key :foo/baz, :key :foo/baz}]
The root query is:
[([:foo/by-id 0] {:other-data other-data})]
#2020-01-1419:47wilkerlucio@U0BR5D7A6 I believe you are getting into this: https://github.com/wilkerlucio/pathom/issues/93#2020-01-1505:30levitanong@wilkerlucio that’s not exactly the case. I don’t even get the params in my resolver. Also, the ast I get is for a property of one of the outputs, not the join around all of them. So it seems like the wrong ast is being passed to the resolver.#2020-01-1608:42cjmurphy@U0BR5D7A6 You can use a plugin to make sure every resolver gets the parameters you pass. It is part of Fulcro RAD but I'll copy it in here. You access them as query-params .#2020-01-1608:42cjmurphy
(def query-params-to-env-plugin
  "Adds top-level load params to env, so nested parsing layers can see them."
  {::p/wrap-parser
   (fn [parser]
     (fn [env tx]
       (let [children (-> tx eql/query->ast :children)
             query-params (reduce
                            (fn [qps {:keys [type params] :as x}]
                              (cond-> qps
                                      (and (not= :call type) (seq params)) (merge params)))
                            {}
                            children)
             env (assoc env :query-params (dissoc query-params :pathom/context))]
         (parser env tx))))})
#2020-01-1610:04levitanong@U0D5RN0S1 thanks! your tip worked like a charm.#2020-01-1416:22Aleedreading documentation on resolvers, wondering if I should be wary of automatic data dependency resolutionĀ  i.e.Ā in docs, this snippet [{::latest-product [:product/title :product/brand-id]}] ::latest-product doesn’t return :product/brand-id directly, but because it returnsĀ `:product/id` pathom can use product-brand resolver to get :product/brand and then brand-id-from-name to get :product/brand-id in contrast with graphql resolvers, you declare how a property can be retrieved explicitly. Which is less powerful, but I feel like things can break easily if you're not exact about what depends on what. wondering if this is an issue ppl experience with pathom, or if I just need to change my mental model to accommodate the automatic data dependency resolution#2020-01-1416:56Chris O’DonnellI think that transitive dependency resolution is what makes pathom really shine. It lets you decouple the fetching of different attributes of an entity. I have not used pathom in production, only in toy projects, so perhaps others will have more nuanced perspectives. Most of my resolvers take an ID as input and return a set of attributes for that entity, possibly including a join to another entity with that entity's ID. With that setup it's pretty clear where attributes are coming from. You can also use pathom trace to examine what pathom did to resolve your query if you're ever confused.#2020-01-1420:29Aleedwasn't aware of pathom trace, thanks for pointing that out#2020-01-1419:51wilkerlucio@alidcastano hello, like @codonnell said, this is the killer feature for Pathom, I've been doing it for some time, and so far this hasn't been a problem, the biggest system using it that I worked on had close to 2000 different attributes, when you start to get to this size, what I see is that there are some "hub attributes" (usually ids, that correlate things) while most of attributes appears only once in the resolvers, since you don't actually get many options for the same thing (altough it is supported) it's usually easy to figure how something got there, and the tools are also there to help with it, makes sense?#2020-01-1420:28Aleedhaven't played around with code yet, just reading docs, so only other question I have is - if you try to access a property that can't be resolved, what happens? would be nice to have a compiler similar to Relay that warns you so you don't encounter an error at run time#2020-01-1421:28pithylessIn practice you have tools like https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/exploration.html to visualize all the ways certain attributes are reachable. These indices are also used by tools like https://github.com/fulcrologic/fulcro-inspect and https://github.com/wilkerlucio/pathom-viz to implement auto-complete for interactive query building (if it's not auto-completing, it means you're writing a query that won't resolve). I suppose you could use these same indices and tools to write similar kind of linters for compile/runtime.#2020-01-1606:32cjmurphyAnyone seen this error message before? How best to get some better context for where it came from (without altering the source):
Execution error (IllegalArgumentException) at com.wsscode.pathom.connect/split-good-bad-keys$fn (connect.cljc:643). => Don't know how to create ISeq from: clojure.lang.Keyword
#2020-01-1606:48cjmurphyIt turned out that I wasn't returning the correct format from a resolver. I was (for reasons unknown!) returning a vector of vectors rather than a vector of maps at a particular join.#2020-01-1615:43fjolne@U0D5RN0S1 @U066U8JQJ it looks like the latest Pathom release (one of, at least) became more strict on what is allowed to be returned. I did return true in a few mutations and it stopped working. Not a big deal ofc, jfy#2020-01-1610:56thosmosI’m looking into moving an existing javascript UI to query a pathom/eql/transit api instead of graphql. Has anyone done this with transit-js? … Well, that was easy enough … I got the basic idea to work for reading the result, something like:
var t = require("transit-js")
var r = t.reader("json" , {
// , {
//   "handlers": {
//     "key": function(rep) { return rep.toString() }
//   }
   mapBuilder: {
    init: function(node) { return {} },
    add: function(ret, key, val, node) { 
    let k = key;
    let v = val;
    
    if( t.isKeyword(key) )
        k = key.toString();

    if( t.isKeyword(val) )
        v = val.toString();
    ret[k] = v;
    return ret },
    finalize: function(ret, node) { return ret }
  }
}
);

var res = r.read('["^ ","~:org.riverdb.db.agencylookup",[["^ ","~:db/id","17592186158592","~:agencylookup/AgencyCode","AMS"]]]');

=> 

{":org.riverdb.db.agencylookup": [{":agencylookup/AgencyCode": "AMS", ":db/id": "17592186158592"}]}
#2020-01-1623:27cjmurphyI'm still trying to understand when transitive dependencies are picked up. For example I'm supplying :organisation/id as an input to a resolver. With that attribute alone a lot of other attributes are determined. For example given an organisation you get a time-info which has a current-period. So should I be able to rely on Pathom to work out :time-info/current-period by itself? If so the query could supply the :organisation/id input, but :time-info/current-period would just be stated as a 'co-input' to the resolver, and Pathom would find the current period of the organisation and put it in. What I'm seeing is that as soon as I put :time-info/current-period in as an input the resolver is not called.#2020-01-1700:07wilkerlucio@cjmurphy hello, the way pathom resolve transitive deps is by walking the index, with reader2 and parallel-reader, they use a compute-plan function which figures the paths available to go from the data you have to the one you asked, it walks in a reverse way, trying to think of your example, this is what your resolver looks like?
(pc/defresolver org-data [_ _]
  {::pc/input  #{:organisation/id}
   ::pc/output [{:organization/time-info
                 [:time-info/current-period]}]}
  ...)
#2020-01-1701:23cjmurphyNo the org-data resolver output just has
[{:organization/time-info
                 [:time-info/id]}]
as the output, amongst other things. There's another resolver for time-info that goes from :time-info/id to all the time-info scalar attributes, including current-period.
#2020-01-1701:26cjmurphyAlso my readers are just these: [p/map-reader pc/reader2 pc/open-ident-reader], so no parallel-reader .#2020-01-1703:38wilkerlucioyeah, the reason why you can't ask for :time-info/current-period is that Pathom never switch context, so, there are sibling and nested dependencies, in your case there is a nesting, so since you may have multiple links like this (returning same properties for different entities) Pathom will not try to pull data from nested resources#2020-01-1703:39wilkerluciothat said, if you want to make that possible, one extra resolver can do it, like:
(pc/defresolver organization-time-data [env {:organization/keys [time-info]}]
  {::pc/input  #{:organization/time-info}
   ::pc/output [:time-info/id
                :time-info/current-period]}
  time-info)
#2020-01-1703:40wilkerluciobut if that's the case (you don't have multiple assotiations, being a simple to-one relationship), then your original resolver for organization may already return that flat, if you do so then it just works, makes sense?#2020-01-1704:31cjmurphyI think so. So where I need to nest from organisation just do so in org-data resolver's output, as you did in your first code snippet. So along with :time-info/id there will also be :time-info/current-period . Then Pathom will understand that that nesting can be done. I'll try that. Are 'subling' dependencies fine? Still not sure the definition of that. Still getting to grips with Pathom, not there yet...#2020-01-1706:52cjmurphyIn the end I went with the idea of an extra resolver. Not exactly the one you put but an 'id-to-id' resolver, called org-id->time-info-id-resolver. Some kind of Pathom-intuition (more blind luck than understanding).#2020-01-1712:28wilkerlucioI meant sibling*, sorry, typo, hehe šŸ™‚#2020-01-1712:29wilkerluciohave you seen this? https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/indexes.html#_index_oir#2020-01-1721:21cjmurphyNo. Looks good. I'll be giving it a read today. So far I've only been reading the docs here: https://wilkerlucio.github.io/pathom#2020-01-1918:40thhelleranyone know a public EQL server?#2020-01-1918:40thheller@wilkerlucio do you have your youtube/spacex stuff your used in your talks public somewhere?#2020-01-1918:41wilkerluciothey are available, but very incomplete: https://github.com/wilkerlucio/pathom-connect-spacex https://github.com/wilkerlucio/pathom-connect-youtube#2020-01-2203:50cjmurphyA pc/defmutation takes two arguments: env and a second argument. Is it true that the second argument would be defined as 'query params' rather than 'input'? I'm thinking it would make sense to have 'input' as well (so a third argument). That way the caller could provide the mutation with different inputs to that which the mutation needs, much the same as can be done when calling resolvers.#2020-01-2213:48wilkerlucio@cjmurphy fun you mention it now, I've been recently playing with a similar idea, but instead of adding a new param, I made a transformer that automatically resolves the mutation params from the user given params, its like getting what the user sent, use as context to run what the query in ::pc/params, and them send that resolved result as the params#2020-01-2213:48wilkerlucionot finished, but the current code:#2020-01-2213:48wilkerlucio
(defn resolve-mutation-params
  [mutation]
  (update mutation ::pc/mutate
    (fn [mutate]
      (fn [{:keys [parser] :as env} params]
        (go-catch
          (let [params-eql      (::pc/params mutation)
                resolved-params (->> (parser
                                       (-> env
                                           (assoc ::p/entity (atom params)))
                                       params-eql) <?
                                     (p/elide-items p/special-outputs))]
            (<?maybe (mutate env resolved-params))))))))

(pc/defmutation delete-todo [env doc]
  {::pc/sym       `todo/delete
   ::pc/params    [::pdb/id ::pdb/rev]
   ::pc/transform resolve-mutation-params}
  (pdb/remove-doc env doc))
#2020-01-2214:08cjmurphyI like the idea of the caller being explicit about the difference. Like you say in the docs a 'query param' (that's Tony's term from his plugin used in RAD, to give params to all resolvers) is not involved with dependencies, is used for sorting and things like that, whereas an input is about dependencies and is more commonly used. At the end of the day a mutation is not unlike a resolver, and a resolver has both ideas, so for consistency a mutation could also have both ideas?? I mentioned in the Fulcro channel that multiple resolver inputs did not seem to be formally supported by Fulcro. I would guess that people sometimes use params where they would be better off using inputs.#2020-01-2216:59wilkerlucioaltough I agree they are similar, in pratice params are way more common in mutations than in resolvers, and that's I think its big enough to worth the difference in usage, so I think I'll stick to this idea for the library, but in the same way I'm creating that transform, you can create your own that behaves in the way you described, it can even add the third argument you mentioned#2020-01-2217:01cjmurphyCool thanks šŸ™‚#2020-01-2222:40Ī»ustin f(n)Hi, I am trying to use pathom on the server side. Is there a straightforward way to setup Pathom Viz tools in a way that it can work from a parser that hasn't been built with CLJS in mind?#2020-01-2301:40wilkerlucio@austin021 not yet, I've been working on a solution to this exact problem, but I can't promise any timeline, the work is been done in this branch: https://github.com/wilkerlucio/pathom-viz/tree/standalone-electron, the idea is to a have a standalone electron app that you can connect parsers directly, until that's ready the best is to setup a fulcro app just enough that you have access to the tools check šŸ‘‡#2020-01-2317:09Ī»ustin f(n)I am glad to see there is progress on it! Do you know how close your solution is? Perhaps close enough I can finish getting it working for me with my little-to-no knowledge of fulcro?#2020-01-2417:03wilkerlucionot so easely, there are some decisions to be made and new UI to support some things like multiple connections, would require some Fulcro knowledge at this point, thanks for the offer#2020-01-2311:20kszabo@austin021 there is actually this way of doing for now: https://github.com/wilkerlucio/pathom-viz/blob/master/docs/standalone.adoc#2020-01-2311:23kszabojust add these files: https://gist.github.com/thenonameguy/851d3ad5edf14c2246cc9bb06f8433da#2020-01-2311:24kszaboand you get a combined version of pathom-viz + query editor#2020-01-2311:24kszaboyour application should support CORS requests though#2020-01-2318:16Ī»ustin f(n)Thanks @thenonameguy! That worked great! To anyone else who wants this, I simply have a resolver connected to a 'POST' endpoint that receives/sends transit. The only thing I had to add was a step to filter out 'fn' objects from the response so transit could serialize it.#2020-01-2318:17kszaboyup, or you can write a custom transit serializer which Wilker documented in the docs. (I’m in favor of just stripping fns from the index in the pathom-viz resolver)#2020-01-2417:41Ī»ustin f(n)Question about data modeling with resolvers: I already know how to use the vector-collection format (or whatever it is called) for output.
::pc/output  [{::s-coach/sessions
               [::sf/id
                ::user/email
                ::s-coach/first-name
                ::s-coach/last-name
                ::user/visitor-id
                ::story/name
                ::session/id
                ::session/start-time
                ::session/location-url]}]
However, now I am building a resolver to enclose a more maplike existing data structure. e.g.
{:key1 {:a 1
        :b "foo"}
 :key2 {:a 3
        :b "bar"}
 :key3 {:a 14
        :b "baz"}}
Is there a way to simply specify the output of this? Or do I need to restructure things to:
[{:id :key1
  :a 1
  :b "foo"}
 {:id :key2
  :a 3
  :b "bar"}
 {:id :key3 
  :a 14
  :b "baz"}]
And use
::pc/output [{:my-collection
              [:id :a :b]}]
?
#2020-01-2418:16wilkerlucio@austin021 hello, there is a feature to help with that coming on the next version, the ::pc/output will remain the same (as you did), but you have to signal to Pathom that this map should be threated like a "map of maps", the distinction is nescessary because otherwise pathom would try to run the sub-query against the map directly (what you see happening now). this is an example of how to do use this:
(pc/defresolver some-resolver [env {:keys []}]
  {::pc/output  [{::s-coach/sessions
                  [::sf/id
                   ::user/email
                   ::s-coach/first-name
                   ::s-coach/last-name
                   ::user/visitor-id
                   ::story/name
                   ::session/id
                   ::session/start-time
                   ::session/location-url]}]}
  {::s-coach/sessions
   ; add this metadata to the map, to signal map of maps to Pathom
   ^::p/map-of-maps
   {:key1 {:a 1
           :b "foo"}
    :key2 {:a 3
           :b "bar"}
    :key3 {:a 14
           :b "baz"}}})
#2020-01-2418:16wilkerlucioyou can try that using pathom 2.3.0-SNAPSHOT, but this version may be unstable (not production tested, and a lot of changes there), use at your own risk#2020-01-2504:00wilkerlucio@austin021 I was playing with map-of-maps, but was pretty broken, I just updated the snapshot, this one includes some fixes for it, should work on the latest snapshot#2020-01-2709:55roklenarcichm the example here: https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/graphql/fulcro.html doesn’t seem to work because namespaces like fulcro.client.network don’t exist#2020-01-2713:24Bjƶrn EbbinghausHm. The examples are using fulcro 2 ... This needs to be updated. šŸ˜‰#2020-01-2714:06wilkerlucio@roklenarcic yeah, for F2, I don't have a shared library with that yet, but you can use this (rename the ns to something that makes sense to you):
(ns remote-pathom
  (:require [edn-query-language.core :as eql]
            [com.wsscode.common.async-cljs :refer [<?maybe]]
            [com.fulcrologic.fulcro.algorithms.tx-processing :as txn]
            [cljs.core.async :refer [go]]))

(defn pathom-remote [parser]
  {:transmit! (fn transmit! [_ {::txn/keys [ast result-handler]}]
                (let [edn           (eql/ast->query ast)
                      ok-handler    (fn [result]
                                      (try
                                        (result-handler (assoc result :status-code 200))
                                        (catch :default e
                                          (js/console.error e "Result handler for remote failed with an exception."))))
                      error-handler (fn [error-result]
                                      (try
                                        (result-handler (merge error-result {:status-code 500}))
                                        (catch :default e
                                          (js/console.error e "Error handler for remote failed with an exception."))))]
                  (go
                    (try
                      (ok-handler {:body (<?maybe (parser {} edn))})
                      (catch :default e
                        (js/console.error "Pathom Remote error:" e)
                        (error-handler {:body e}))))))})
#2020-01-2716:59souenzzoA Small lib to qualify/unqualify data with #pathom Docs still WIP. https://github.com/souenzzo/eql-as#2020-01-2719:04mssI’m trying to dispatch a server-property load! call in fulcro to a pathom resolver with params. the server property seems to be getting picked up properly, but the params I added in the options map of load! don’t seem to be getting parsed by pathom and fed into my resolver as params. any suggestions for how to debug?#2020-01-2719:12cjmurphyThere's a Pathom plugin from Fulcro RAD that will fix this. This one is just slightly altered:
(def query-params-to-env-plugin
  "Adds top-level load params to env, so nested parsing layers can see them."
  {::p/wrap-parser
   (fn [parser]
     (fn [env tx]
       (let [children (-> tx eql/query->ast :children)
             query-params (-> (->> children
                                   (reduce
                                     (fn [qps {:keys [type params] :as x}]
                                       (cond-> qps
                                               (and (not= :call type) (seq params)) (merge params)))
                                     {}))
                              (dissoc :pathom/context))
             env (assoc env :query-params query-params)]
         (parser env tx))))})
#2020-01-2719:14cjmurphyThen you can pick up the key query-params from in the env of the resolver you want to see your parameters in.#2020-01-2813:23ak-coramHi, I have a working resolver and I'm trying to transform it to work with batches. It all works as expected, but sometimes my resolver gets called with nil as the input. This can lead to :com.wsscode.pathom.core/not-found appearing in the results (for entities where the required identifier for my resolver is already present). It seems to happen randomly and sometimes I get the expected full result, which makes this pretty hard to debug :) Is there maybe something I have to return from the resolver to signify that the input is inadequate? The resolver is executing multiple batches in a single query if that helps. Also I'm using the parallel-parser, so this might be some kind of race condition. I wanted to ask here before I try to work on reproducing with a minimal example. Any ideas?#2020-01-2813:27kszaboCreating a reproducible example first is always helpful, even for your own understanding šŸ™‚#2020-01-2813:29ak-coramI'll try, thought I may be missing something obvious. I just hope it's not a race condition, because it might be tricky to reproduce with a small example.#2020-01-2813:35souenzzo@ak407 you are using :>/things ?#2020-01-2813:39ak-coram@souenzzo: I'm not sure what you mean, so I'm probably not šŸ™‚#2020-01-2813:46souenzzoIt' about that. I've had problems https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/core/placeholders.html#2020-01-2813:44souenzzo;; (require '[com.wsscode.pathom.parser :as pp]) try to add ::pp/max-key-iterations to your env. The default value is 5 , you can try maybe 20 I'm not sure if it goes into (p/parser {...}) arg or in (parser {...} [...]) context. So you can put in both šŸ™‚#2020-01-2813:53ak-coramthanks, I'll try it and get back to you in a minute#2020-01-2813:59ak-coramit doesn't seem to have an effect, resolver is still called varying number of times with nil as the input leading to a number of "not-found" entries in the result#2020-01-2814:00ak-coramI've also tried reproducing with a minimal example, but it seems to work fine#2020-01-2814:01ak-coramI'll try introducing random delays into the resolvers to simulate an actual api call#2020-01-2814:28ak-coramno luck so far šŸ˜•#2020-01-2814:54ak-coramI think it has something to do with the cache in the parallel-batch function (connect.cljc)#2020-01-2814:55ak-coramit seems to work fine when I don't get cache hits#2020-01-2814:55ak-coramI've tried disabling caching on the resolver, but that kills the batch functionality too#2020-01-2814:56ak-coramfun thing is: there shouldn't be any cache hits for this query as all batched items are unique#2020-01-2815:14ak-coramfalse alarm, it just randomly worked for several queries. not sure if it has anything to do with the cache or not#2020-01-2815:31ak-coramI think I'm getting closer#2020-01-2815:40ak-coramso it looks like parallel-batch is sometimes called twice for the same batch. (remove #(p/cache-contains? env [resolver-sym (second %) params])) seems to remove all entries from one of the batches (which I assume to be the second one), hence the resolver is called with nil. if I remove that line from parallel-batch the resolver is called twice with the same arguments, but at least I consistently get results back.#2020-01-2817:29wilkerlucio@ak407 good catch, I guess you are writring, it shouldn't exclude the cached ones, just read from the cache, I can see that being a miss, do you think you can make a minimal example and open an issue about it?#2020-01-2819:32ak-coram@wilkerlucio: I'm having some trouble reproducing it with a minimal example, but will try to come up with something tomorrow. thanks everyone for helping out#2020-01-2820:44Ī»ustin f(n)In the pathom viz tools when you have multiple inputs in a resolver, the 'input' key is considered the set of the multiple inputs. Makes sense, but when you then try to inspect the full graph the resolvers with multiple inputs are not linked in any way to their inputs. Are there any plans to add something to the graph visualization to show that multiple-input resolvers are related to their inputs somehow?#2020-01-2900:28wilkerlucio@austin021 this is a matter of figuring a good way to visualize that, I agree this can be improved, just did bother enough to spend the time on it. one thing is that on the attribute listing, you can see with which attributes it is combined, that information is directly acessible on the index, so the data is there#2020-01-2914:01ak-coramHi, if anyone's interested: I've submitted a PR for fixing the issue I've encountered yesterday (see above). https://github.com/wilkerlucio/pathom/pull/141#2020-01-2914:52wilkerlucioawesome, thanks! I’ll take a look on it later today#2020-01-2915:21ak-coramsounds great, let me know if you have questions#2020-01-2915:40tony.kay@wilkerlucio have you had any discussion about putting docstrings on the indexes?#2020-01-2915:40tony.kaythat way Inspect in Fulcro could show docs about attributes as you explore the graph…or we could create an API doc tab#2020-01-2915:41tony.kayseems a simple-enough add#2020-01-2915:43tony.kaywould also then be simple to make some generalized tool that could hit a pathom endpoint and generate a live documentation site.#2020-01-2915:45wilkerlucio@tony.kay if I remember right that was already implemented for resolvers#2020-01-2915:45kszaboI have been toying with this idea, combined with some form of serializable specs. There is support for docstring, the example index-explorer resolver should be updated to include it, that should be all: https://github.com/wilkerlucio/pathom/blob/c30ccb4d875ec2adbd157905d84e53e9d72f5129/test/com/wsscode/pathom/connect_test.cljc#L228#2020-01-2915:45wilkerluciobut mutations may be missing#2020-01-2915:46tony.kaygreat, yeah, I didn’t see it in docs anywhere#2020-01-2918:08sifI have resolvers that only run once after server restart or (tools-ns/refresh) but then it doesn't run a second time (I have to restart the server to run "once"), any help please, I'm using fulcro web template?#2020-01-3017:48Ī»ustin f(n)Can you provide multiple 'idents' to the root of a query? If so, how? For example, if you want to test a resolver that requires two inputs. For example, this is what I have tried so far... [{[:input1 "foo" :input2 42] [:product]}]#2020-01-3017:48Ī»ustin f(n)The only other place where I currently am using resolvers that take two items, one of the items is a global. I have had no issues with using that one.#2020-01-3017:51cjmurphyYes, you have to use multiple inputs: https://wilkerlucio.github.io/pathom/#_multiple_inputs#2020-01-3018:07kszaboI submitted an improvement idea about this, see the thread for some points on this use-case: https://github.com/wilkerlucio/pathom/issues/140#2020-01-3018:47kszabo@austin021 you either have to create a composite key attribute like [{[:input1+input2 ["foo" 42]] [:product]}] or do [{([:input1 "foo"] {:pathom/context {:input2 42}}) [:product]}]. My idea was to enable [{{:input1 "foo :input2 42} [:product]}] which I still believe is nicer syntax so I’ll probably release a tiny lib to enable it if you want to enable it in your parser.#2020-02-0301:19kszaboBit more cleaning up to do, sadly this required a fork of eql+pathom#2020-01-3116:29kszaboAfter a bit more thought: FYI for Fulcro users this also wouldn’t break client-side normalization, as your :ident fn has all of the props available which you queried for: i.e: if you want to have an Account+User component, which combines :account/id and :user/id for instance to enable a :user/id to belong to multiple accounts. And you don’t want to create ā€˜syntethic’ ident handling in Pathom like [:account+user/id [42 69]] then you can write :ident (fn [] [:account+user/id [(:account/id props) (:user/id props)]]) . This way the combination of the system attributes to support client-side normalization stays on the client/Fulcro side, as it should.#2020-01-3116:44kszabo(df/load this {:account/id 69 :user/id 42} Account+User) is how data-fetching would look like#2020-01-3117:28kszabothis being said I’m more interested in the maximal graph vision of pathom for microservice graph apis and the potential for RAD to visualize them#2020-01-3117:29kszaboand dynamic queries by humans for setting initial contexts with multiple attrs to query more of the graph#2020-02-0100:40Ī»ustin f(n)I am noticing that resolvers with multiple inputs sometimes try to resolve even when all the inputs are not present. Is there a way to require every input? In my case, I have some simple resolvers that take a bunch of keys and put them into a map under a new keyword. If there is 'enough' attributes it returns it with the keys it had, but if there is only one or two of them it returns :com.wscode.pathom.core/not-found#2020-02-0101:19fjolneare you sure they don’t try resolve when the inputs just have nil values? that’s different from absence#2020-02-0321:03Ī»ustin f(n)Pretty sure. I put logging in the resolver, and the input to the resolver is missing one of the values entirely even though the inputs has it listed.#2020-02-0321:05Ī»ustin f(n)I am trying to put together the simplest thing that replicates the effect so far. The actual example has lots of interconnected resolvers...#2020-02-0321:18Ī»ustin f(n)I just noticed that the resolvers follow the 'correct' path that I expected it to, one that provides the missing field. But it only does this on the first resolution. The correct path is fairly slow, so I wonder if the path weighting algorithm is trying to get around resolving the slow resolver.#2020-02-0321:56Ī»ustin f(n)Alright, I looked up how the weighting works. After noticing that "If a resolver call throws an exception, double it’s weight", I decided to change my resolver to throw an exception if any of the inputs are missing.#2020-02-0321:57Ī»ustin f(n)This fixed the behavior, as pathom is now correctly falling back to the other path on failure, as well as discouraging the problem path somewhat.#2020-02-0100:42Ī»ustin f(n)This is becoming problematic because this not found is happening even when there is an alternate path to get to the value I am asking for, but the 'composite' resolver is prioritized or something. Maybe I am simply modeling the data relationships incorrectly.#2020-02-0100:51kszaboAll inputs are required for resolvers until https://github.com/wilkerlucio/pathom/issues/70 gets implemented. For the not-found issue, I strongly suggest using the com.wsscode.pathom.core/elide-special-outputs-plugin, that removes that from the final set. Your resolvers should never see that value.#2020-02-0117:05Ben GrabowCan I get a little help understanding how to-many resolvers and queries are supposed to work? I'm trying to add pathom on top of the Lacinia tutorial project and I'm stuck on getting the board game -> designers query to work.#2020-02-0117:06Ben GrabowHere's the query I'd like to work https://github.com/bgrabow/clojure-game-geek/blob/6866af5668a520907809993592b2761fc8b733e4/src/clojure_game_geek/pathom/parser.clj#L26#2020-02-0117:07Ben GrabowI want to look up all the designer names for a given board game ID. Right now I'm getting a reader-error:
(a/<!! (parser {:database {:data (-> (io/resource "cgg-data.edn")
                                     slurp
                                     edn/read-string
                                     atom)}}
               [{[:board-game/id "1234"]
                 [
                  {:board-game/designers [:designer/name]}]}]))
=>
{[:board-game/id "1234"] #:board-game{:designers :com.wsscode.pathom.core/reader-error},
 :com.wsscode.pathom.core/errors {[[:board-game/id "1234"] :board-game/designers] "class java.lang.IllegalArgumentException: find not supported on type: java.lang.String"}}
#2020-02-0117:12Ben GrabowI can make the board game id -> designer ids jump, and I can do the designer id -> designer name jump, but I must have something hooked up wrong or I'm misunderstanding how the designer ids -> designer id jump is supposed to work:
(a/<!! (parser {:database {:data (-> (io/resource "cgg-data.edn")
                                     slurp
                                     edn/read-string
                                     atom)}}
               [{[:board-game/id "1234"]
                 [:board-game/designers]}
                {[:designer/id "200"]
                 [:designer/name]}]))
=> {[:designer/id "200"] #:designer{:name "Kris Burm"}, [:board-game/id "1234"] #:board-game{:designers ("200")}}
#2020-02-0117:25Chris O’DonnellIt looks to me like your board game resolver is returning {:board-game/designers ("200")} where it should be returning {:board-game/designers ({:designer/id "200"})}.#2020-02-0117:40Ben GrabowYeah! That's working now, thank you! https://github.com/bgrabow/clojure-game-geek/commit/5761218b59e612e192d2fda88124bebad1c89658#diff-bdf581a0bb704397fed938b7d562328d#2020-02-0117:42Ben GrabowIs there a way I can query for the designer names without knowing about the :board-game/designers key? I.e. I know that a board game has designers who have names, but I don't want to care about the implementation details of how that path gets traversed.#2020-02-0118:11Ben GrabowHmm, maybe I'm thinking about that wrong. If we imagine a schema where a board game has some designers who authored the game, and some other designers who advised the design, I would need to specify if I want to look up the names of the authoring designers vs looking up the names of the advising designers. So the relationship represented by :board-game/designers is a necessary part of the query specification.#2020-02-0120:34Chris O’DonnellRight, that is what's connecting the board game with its set of designers, and that connection has to live somewhere. There are other ways to model it, but this one seems pretty natural to me. šŸ‘#2020-02-0301:22kszabois there a reason why https://github.com/edn-query-language/eql is not used for eql parsing inside pathom? potemkin/import-vars could be used to not break p/ast->query users for instance#2020-02-0315:01wilkerlucioI think right now is a bit mixed, but since they have no difference it should not affect anything, did you find a place where they diverge?#2020-02-0315:07kszaboI replaced the predicate based dispatch to protocol-based one in my eql idea pr to improve hot path performance: https://github.com/edn-query-language/eql/pull/9/files#diff-9fe21494d17308f0d9211266c0baa17dR344 I thought eql was the definitive place to do this, I was surprised that pathom had an identical impl. That’s all, nothing major#2020-02-0316:10wilkerluciothe reason for duplication is only because the code was written in pathom before eql existed#2020-02-0316:11wilkerluciobut cool ideia, did you measure the performance impact with those changes?#2020-02-0316:14kszabonothing too extensive, I’m running integration+load tests currently through the whole parser, trying to get parser overhead down for a 2ms p99 redis resolver so that the whole parser doesn’t take 10ms#2020-02-0316:15kszaboalso trying to reduce GC pressure, shenandoah GC helped a bit#2020-02-0316:17kszabohaven’t ran criterium benchmarks yet, but this way should be faster. @U055NJ5CC had a good talk about achieving high perfomance for reitit at clojure/north, trying to improve perf. Also I started reviewing your 2.3.0-DEV branch, the new reader looks good!#2020-02-0317:49wilkerlucionice, I'm willing to merge performance improvements back to EQL, we just need to make sure things are really faster and they output the same results šŸ‘#2020-02-0318:35kszaboyup, still experimenting with it, for now I’ll keep that PR open until I get to stable state, then I’ll post benchmarks. Feel free to copy stuff over in the meantime if you want. I also dropped dynamic vars as they kill performance and implicit < explicit in my books anyway. I’ll post clj-async-profile graphs as well once I’m done.#2020-02-0318:36kszaboI highly recommend you adopt clj-kondo if you haven’t yet, I found many-many lintable small issues in EQL/Pathom, unused aliases and such.#2020-02-0301:22kszaboor just regular old defn’s#2020-02-0320:59kszabo@wilkerlucio looks like I will have to implement optional attributes in resolvers as well šŸ™‚ Just posting here in case you have another idea, with join-context’s I can write this query:
[{#:myservice.input
    {:email               "
#2020-02-0321:01kszabothere are more attributes that can be inputs, and there are multiple groupings of ids, each with a separate resolver (of course there are ident based resolvers as well for email->email related statistic ids). The input statistics are not only looked up by themselves, but there are cross checks as well (email + ip) for instance. Therefore it’s nicer for a remote service/Pathom parser to send this initial context. The issue as that the resolver :myservice.input-statitistic.groups/all currently has to depend on all of the attributes. Do you see of any other way of easing this restriction?#2020-02-0321:04kszaboof course I can create a global resolver with no inputs, and fetch the entity for the join context to run what I want, but that’s not really semantic. So optional attribute resolvers seem like the only other way#2020-02-0321:04kszabothe other way would be require clients to always send all inputs, with nil values, but that’s not forward-migratable API design#2020-02-0321:21pithylessWhy not transform the query on the server, before passing it onto the rest of pathom processing? Here, you can merge with default optional nils. Client would never need to know.#2020-02-0321:23pithylessOptional attributes in resolvers would break ease-of-understanding when parsing/manipulating the graph (e.g. other tooling, index explorer, etc.), because optional inputs essentially introduce a combinatorial-if and turn your static graph search into a dynamic/conditional one.#2020-02-0322:00kszaboI don’t see a clean place to put this nilling logic. The groups/all resolver is no-go, since it won’t be called. I can cook up another attribute like :myservice.default-input/email that can be provided globally or from :myservice.input/email via an alias2. Then resolvers can hard-depend on the default-input attributes instead.#2020-02-0322:01kszaboThese are my best ideas, still these introduce attributes in the system that are incidental, that only exist because Pathom has these implementation details/choices/limits.#2020-02-0322:07pithylessI was thinking more along the lines of when initializing the pathom parser, add a custom middleware to ::p/plugins that will modify your query if you detect groups/all (e.g. via clojure.walk or specter)
#2020-02-0322:08pithylessBut maybe I misunderstood the problem#2020-02-0322:08kszaboI think resolvers having an optional component don’t complicate matters too much. There could be ::pc/opt-in set. Any existing tooling code before updating will see the regular required attributes (if any). The parser just has to be modified to fetch these keys from the entity and merge them together with the required attributes to pass in as params . The question might come whether to resolve global optional attributes for a resolver that are not in the context if possible, but this can be a parser construction-time/`:pathom/realize-opts #{:opt/foo}` param thing.#2020-02-0322:09kszaboI also though of the plugin route, but in my eyes those should be general and not parser specific. Also server-side rewrites of queries is a slippery slope from an understanding standpoint. I would like to keep the solution in the realm of resolvers if possible.#2020-02-0322:13kszaboThere is the efficiency argument as well, associng tons of nils into the entity atom map just for the sake of some limitation in the parser seems wasteful. Pathom already generates tons of GC pressure in high throughput scenarios, I would not want to make matters worse.#2020-02-0323:00wilkerlucioyeah, I have an idea on my mind on how to do the optionals, they will be part of the new planner thing, but I'm working on more fundamental features on it before (supporting the things that are already supported), the new planner is the thing I described in the Maximal Graph talk, with much better dynamic resolver support, the new planner is way more sophisticated than the current one (the current one can only trace a path for a single attribute, the new all considers all of sibling attributes at once), so the syntax is already defined as well (just don't work) will be something like this: ::pc/input #{:required (p/? :optional)}, the planner can take this in consideration, as the runner, I think its a not a big problem of explosion because in real systems there isn't usually a lot of options to the same output (at least so far)#2020-02-0323:00wilkerlucioif anybody is curious, this work is happening here: https://github.com/wilkerlucio/pathom/compare/query-planner#2020-02-0323:01wilkerluciomost of it is working already, and currently I'm working on tools (given the complexity it can happen inside, without tools is quite hard to follow whats going on)#2020-02-0408:08Krisztian SzaboHave you considered using something like spec/keys? The ability to use (or) and (and) inside :req/opt would make it more sophisticated while not increasing the complexity significantly. You can always parse outs input spec objects into indexes the same way to suit the query planner.#2020-02-0410:29wilkerlucioin the beginning of pathom they were supported, but the lack of nesting for outputs was a problem, and in the end not having the control didnt worth it, so I prefer to keep a custom definition way, in truth inputs also needs to support nesting eventually#2020-02-0410:38kszaboThanks, good 2 know. Maybe with spec2/malli things will change#2020-02-0410:40kszaboMy issue with the (p/? :sys/attr) optional proposal is that it is a breaking change for libraries/tools which only accounted for a set of keywords in ::pc/input.#2020-02-0411:20wilkerluciothats a fair point, I think we can instead use a new name, maybe ::input2, but that feels confusing šŸ˜›#2020-02-0411:34kszabowell if we figure out a way to use specs/whatever contract system to support nested serializable data descriptions, then we could just use that system’s name like ::spec-input/output. I would prefer that to reinventing the wheel for a subset of those system’s functionality. Something like data-specs maybe? https://github.com/metosin/spec-tools/blob/master/docs/02_data_specs.md#2020-02-0411:45kszabocombined with: https://github.com/metosin/spec-tools/blob/master/docs/03_spec_visitor.md#2020-02-0411:45kszaboI think this is worth considering, I’ll try it out if I get the time#2020-02-0412:11wilkerluciobut honestly Im not excited to get specs integrated on the game again, they work for different concerns I think, specially given that spec dont have a stable api, I prefer to dont complect with it#2020-02-0412:13wilkerluciomy motivation for the p/? is that it works nicely with nested structures too, and its a simple evolution on the current one (a mark for optional, keeping all the same)#2020-02-0412:27kszaboI see, fair enough. In my eyes ::input and ::output are very-very close in functionality to fdefs for the resolver fn, just omitting the requirement on the env (which could also be addressed).#2020-02-0413:23wilkerlucio@U08E8UGF7 the design is to make open, if you like some facility like that, I think the way to go is wrap defresolver with your own macro, in that point you can integrate an external source like spec, makes sense?#2020-02-0413:24wilkerlucioor if you are avoiding macros, its all maps in the end šŸ™‚#2020-02-0413:28kszaboYup, I’m glad it’s built up like it is šŸ™‚ I have envisioned this before with ghostwheel/guardrails, that’s why I’m so attached to the idea.#2020-02-0613:42ak-coramis there a way to limit concurrency for the parallel-parser? a large query I have immediately calls a single resolver 900 times. processing in the resolver is limited to a number of threads so most of the calls end up timing out.#2020-02-0614:16kszaboyou should probably write a batch resolver then#2020-02-0614:16kszabohttps://wilkerlucio.github.io/pathom/#_n1_queries_and_batch_resolvers#2020-02-0614:18ak-coramit depends on what query I run though, so I'm not sure I want to make resolver adjustments for individual queries#2020-02-0614:18ak-coramalso it doesn't feel right to control the concurrency in individual resolvers#2020-02-0614:19ak-coramanother issue is that my resolver is already a batch resolver, it just gets called for 900 batches#2020-02-0614:22ak-coramI've tried increasing ::pp/key-process-timeout , but individual resolver calls seem to time out regardless#2020-02-0614:51wilkerlucio@ak407 is this related to the changes you did? can you send some smaller example that demonstrates the problem you are facing?#2020-02-0615:09ak-coram@wilkerlucio I'm fairly sure it doesn't relate to my PR since it's also happening with non-batch resolvers. I'll try to create a small example, but I'll probably only get to it on the weekend.#2020-02-0615:13ak-coramI think I have lots of batches because of my joins: a future parser could probably optimize this too and prepare a single batch across multiple levels of joins (this probably isn't always the right thing to do for performance though)#2020-02-0616:01ak-corammaybe the resolver could even hint at some optimal batch size#2020-02-0617:18kennyDoes Pathom support returning an inputstream from a resolver?#2020-02-0617:55kszabothe values of the attributes are up to you#2020-02-0619:48wilkerlucio@kenny you would have a problem if you try to cross it on a boundary, but if you want to get it and use on the same process, it can work#2020-02-0619:49wilkerluciobut that would just be a value as a input stream, pathom will not do anything special about it#2020-02-1014:56ak-coramHi, I've written a test for a concurrency-related batch resolver issue I'm experiencing when using the current release: https://github.com/wilkerlucio/pathom/pull/141 I'd appreciate it if some people could run it and see if the issue also occurs with their setup. The original test only failed about 50% of the time for me, so I'm running it 5 times within a single test run. The expected outcome should be: fails with master, works with the changes in the PR. Thanks!#2020-02-1021:27eoliphantHi, i’m rrunning into a weird issue where I’m getting ā€œMutation not foundā€ errs in inspect, even though the mutation executed fine on the server. In addition, running the mutation directly in the server repl works fine as well and doesn’t return the error#2020-02-1021:55wilkerlucio@eoliphant this is a inspect bug, the parser on the Fulcro Inspect is not properly reading the mutation responses, the query runs, but you don't get the output back#2020-02-1022:17eoliphantOk cool. Just wanted to make sure it wasn't something I did lol#2020-02-1023:58wilkerlucio[com.wsscode/pathom "2.2.31"] is out! This release adds the fix @ak407 made for parallel batch concurrency issue, thanks!#2020-02-1310:59jaihindhreddySorry for being so wordy. Feel free to ignore if noise. Say I had a query like this that tries to fetch information about the role hierarchy in an organisation:
[{:org/employees [:emp/id :emp/name {:emp/manager [:emp/id :emp/name]}]}]
I'm requesting for all employees, and for each employee. I want their id, name, and the id, name of their manager (the person they report to) In this kind of a situation, the response will end up containing a lot of duplicate information, because of the fact that managers are employees too. Seems like Netflix's JSON Graph solves this problem using their ref concept. Does Pathom do something similar, can we borrow (steal) this idea from JSON Graph, or is this not that big of an issue?
#2020-02-1311:48kszabothis kind of normalization happens in the client-side in Fulcro. There is no current support to reduce the over-the-wire part of this.#2020-02-1316:00Chris O’DonnellI believe transit will compress duplicated values, reducing size over the wire if you use it.#2020-02-1407:15jaihindhreddyOf course, Transit does cache keywords, and that gives us a decent compression, but I'm talking about entire maps that could be repeated dozens (even hundreds) of times, and could be avoided. Is this something Pathom could do in the future? Seems like GraphQL doesn't do this too currently.#2020-02-1411:19souenzzoWith this benchmark
(for [i [1e1 1e2 1e3]]
  (let [baos (.ByteArrayOutputStream.)
        zipper (java.util.zip.GZIPOutputStream. baos)
        data (.getBytes
               (cheshire.core/generate-string
                 {:org/employees (for [i (range i)]
                                   {:emp/id      i
                                    :emp/name    (str "emp" i)
                                    :emp/manager (for [i (range i)]
                                                   {:emp/id   i
                                                    :emp/name (str "emp" i)})})}))]
    (.write zipper data)
    (.close zipper)
    [i (.size baos) (count data)]))
You can see that GZip handles the size when you have duplicated elements The json parser/JS engine should also apply reference/reuse optimizations I'm not sure if reimplement it in query language will result in better performance.
#2020-02-1318:15wilkerlucio@codonnell it does by default for keywords, if I remebember correctly you can extended to introduce extra compression#2020-02-1407:15jaihindhreddydata.fressian is extensible that way but not transit I believe. I couldn't find anything like that on the transit github.#2020-02-1411:50Chris O’DonnellMy mistake, thanks for the correction.#2020-02-1320:02souenzzo[ANN] I wrote some docs to eql-as , a library to qualify/unqualified keywords using #pathom and EQL. If you use pathom and need to handle JSON (aka unqualified keys) checkout. https://github.com/souenzzo/eql-as#2020-02-1407:15jaihindhreddyOf course, Transit does cache keywords, and that gives us a decent compression, but I'm talking about entire maps that could be repeated dozens (even hundreds) of times, and could be avoided. Is this something Pathom could do in the future? Seems like GraphQL doesn't do this too currently.#2020-02-1404:45Ī»ustin f(n)In the Pathom visualizer, what are "Attributes with type mismatch"?#2020-02-1411:38wilkerlucioin this context, the different types are scalar vs link, so for example, if one resolver you have something like ::pc/output [{:user/main-address [:address/id]}] it will consider :user/main-address a link, but if later another resolver says ::pc/output [:user/main-address], that's a scalar, now you have a mismatch, because the data should be consistent, the fix is to find the scalar version and declare what comes inside (or change the name in case they are really different things), makes sense?#2020-02-1418:03Ī»ustin f(n)Alright. So far, I have lots of maps. I have been sometimes using the link notation to drill deeper into their fields, and sometimes not. Should I be saving the link notation for only cases where the results are a collection of items and perhaps some other uses? I.E. When pulling apart a map, should I always simply do so as a scalar + a separate resolver, instead of sometimes using links?#2020-02-1523:26wilkerluciothe more you can inform to pathom, the better, so on every map, if you can know the structure of, better to express it in the ::pc/output#2020-02-1414:29souenzzoThere is a parser plugins that normalizes errors like {:me {:id 1 name ::p/reader-error} ;;=> {::p/error {[:me :name ::p/reader-error]} :me {:id 1}}#2020-02-1519:56eoliphantHi, this is a conditional reader, not a pathom problem but just wondering if anyone knows what might be causing this. I’ve a .cljc file like so
(:require 
  ...
  #
server is fine, but my client shadow-cljs build is breaking, saying that ::pc/output is an invalid kw. can’t for the life of me figure out why the cljs build is even ā€˜seeing’ that
#2020-02-1520:07thhellerits a reader thing. the code must remain readable but if CLJS doesn't have that alias it is not#2020-02-1520:07eoliphantah ok#2020-02-1520:07thhellerbreaks in CLJ too#2020-02-1520:07eoliphantif it goes the other way. gotcha#2020-02-1921:35telekidHowdy! Jumping in this channel because I am curious if anyone has done any work to allow Pathom to act as a GraphQL server.#2020-02-2001:35wilkerlucio@jake142 there is this: https://github.com/denisidoro/graffiti#2020-02-2001:36telekidFaaaaaaascinating. Thank you.#2020-02-2015:14sifHello, I'm sending params through load! params but I'm not seeing the params on pathom side uner :ast any help please?#2020-02-2016:01sifIt's all good. There was a plugin that moved those params under :query-params key in env#2020-02-2017:25BrianWhat am I doing wrong here? I keep getting not-found errors
(pc/defresolver two-args [_ {:keys [num1 num2]}]
  {::pc/input  #{:num1 :num2}
   ::pc/output [:num1Squared :num1PlusNum2]}
  {:num1Squared (* num1 num1)
   :num1PlusNum2 (+ num1 num2)})
#2020-02-2017:25BrianWhich I am then trying to use with the following:
(<!! (parser {} [{[:num1 2 :num2 3] [:num1Squared]}]))
#2020-02-2017:26BrianTo be clear, every time I remove all the num2 references it works perfectly so the problem should be with the additional input#2020-02-2017:39BrianPerhaps I should be using parameters..#2020-02-2017:43BrianBut I feel like you shouldn't use parameters because num1 and num2 are both required by num-args and thus they should be dependencies (inputs) not parameters#2020-02-2018:00BrianI've also tried this mix of inputs+parameters but I get a "Key must be integer" error
(<!! (parser {} [{([:num1 3] {:pathom/context {:num2 2}})
                    [:num1PlusNum2]}]))
but my biggest concern is that if I'm passing inputs in as parameters, aren't I circumventing the whole graph thing that pathom has going for it?
#2020-02-2018:06Chris O’Donnell@brian.rogers check out https://wilkerlucio.github.io/pathom/#_multiple_inputs#2020-02-2019:06BrianI've tried that with this code:
(<!! (parser {} [{'([:num1 3] {:pathom/context {:num2 2}})
                    [:num1PlusNum2]}]))
however I get not-found even with that
#2020-02-2022:17Chris O’DonnellThat'll teach me to try to reply to people on the go. My bad! Not sure what's wrong with the above code without being able to work at a repl.#2020-02-2113:22wilkerlucio@brian.rogers this example looks correct, did you confirmed you have registered the resolver when building the parser?#2020-02-2115:10BrianI'm not really sure what has changed but I've ended up finding a working solution from the docs, then slowly working my way backwards towards my initial attempt until I found what the difference was. I worked my way all the way back and found that there was no difference and no mistakes and my initial attempt worked. Not sure what happened. Gotta love CS lmao#2020-02-2609:01ouvasamI also have the same error "Key must be an integer" using something like :
(<!! (parser {} [{([:catalogue/id 1] {:where {:a 1}})
                   [:catalogue/name
                    :catalogue/status]}])))
#2020-02-2609:01ouvasamdid you found a solution ?#2020-02-2022:59Ī»ustin f(n)Why would a resolver be called without all its inputs? This has been happening fairly often for me, especially when doing deeper queries. I can sometimes 'fix' it by explicitly adding the missing input to the request at the same level, so it doesn't "forget" to get it.#2020-02-2023:13Ī»ustin f(n)Even stranger in this case I have no problem resolving one item :foo . When I ask for a different item that requires JUST :foo , and has no other resolution paths, it takes a different strategy that causes the aforementioned error and fails.#2020-02-2023:13Ī»ustin f(n)As if, by going one step deeper it loses track of something?#2020-02-2112:13rickmoynihannoob question: Is there a pathom ring handler and corresponding cljs client for making pathom queries anywhere? Or do people hand roll this each time; or am I misunderstanding something?#2020-02-2112:53wilkerlucio@austin021 I'm not sure if I understand, a resolver should only be called if the inputs can be satisfied, can you send an example that demonstrates what you are saying?#2020-02-2113:01wilkerlucio@rickmoynihan hello Rick, not sure what you mean, when I'm doing a server I usually roll an endpoint and put the parser to work on it, many times I juts use it in CLJS directly, so there is not need for server setup in these cases, what kind of setup are you looking into doing?#2020-02-2113:21rickmoynihan@wilkerlucio I’m more at the pondering stage right now, than actively looking at implementing anything — but was essentially thinking about having a single endpoint/route to serve EQL requests from cljs front end components#2020-02-2115:01Brian@rickmoynihan maybe this could be helpful to you to see. This is what I do. I am sending my data over in transit form (although it isn't necessary I could just send the raw query. Here is a link to it though https://github.com/cognitect/transit-format). This is in hooked into an Ion since I am running it in the cloud and my query is placed in the body of the request at the :input
(defn untransitize [query]
  (transit/read
    (transit/reader
      (ByteArrayInputStream. (.getBytes query))
      :json)))

(defn eql-query
  "run eql query into pathom and return the transitized result"
  [{:keys [context input]}]
  (let [query (untransitize input)
        response (<!! (parser {} query))
        out (ByteArrayOutputStream. 4096)]
    (transit/write (transit/writer out :json) response)
    (str out)))
#2020-02-2115:03BrianI've then listed the eql-query function in my ion config like so
:lambdas  {:eql-query
            {:fn          my-namespace/eql-query
             :description "run eql query into pathom and return the transitized result"}}
and now I have an endpoint I can just throw eql queries at. In my case they are using transit, but I could very easily just use the raw query. Does this help?
#2020-02-2115:30rickmoynihan@brian.rogers: thanks, that’s exactly the kind of thing I was expecting to do#2020-02-2115:36roklenarcicHm, how do I translate this example query in GraphQL into pathom eql query?
{
  countries(where: {name: {eq: "United Kingdom"}}) {
    cities {
      name
      population
    }
  }
}
#2020-02-2115:37roklenarcicCurrently I have
::pcg/prefix    "eb"
   ::pcg/ident-map {"countries" {"where" :eb.Country/where}}
and then I tried query:
(parser {} [[:eb.Country/where {:name {:eq "United Kingdom"}}]])
#2020-02-2115:45roklenarcicI don’t think I understand this ident map correctly I guess#2020-02-2116:05Brian@roklenarcic I think you are blending edn data and eql queries without meaning to. Or perhaps blending datomic pull syntax and eql queries. That looks to me like you're putting pull syntax into an eql query#2020-02-2116:10BrianI'm imagining a resolver where the input would be a country name and the output would be :population in which case your parser and query call would look something like this:
(parser {} [{[:name "United Kingdom"][:population]}])
here you're saying in the first square brackets that you have an input of :name and you want pathom to give you a :population . This could be used with a resolver like this
(pc/defresolver population-resolver [_ {:keys [name]}]
  {::pc/input #{:name}
   ::pc/output [:population]
   {:population (get-population-function name)}})
and you'd need to write the get-population-function yourself and query your database, likely with pull syntax
#2020-02-2116:10BrianDoes that help?#2020-02-2116:14myguidingstarfrom the query I guess @roklenarcic is trying to use pathom to connect to a hasura graphql server?#2020-02-2116:16roklenarcicI am not mixing. But the where parameter is complex and judging by pathom docs i need to put as the ident#2020-02-2116:16roklenarcicSorry on phone now#2020-02-2116:19roklenarcic@myguidingstar everbase graphql#2020-02-2116:20BrianAh in that case I cannot help. Good luck!#2020-02-2118:22Chris O’DonnellYou need to parameterize your query; it should look something like this:
[{'(:my.prefix/countries {:where {:name {:eq "United Kingdom"}}}) [{:my.prefix/cities [:my.prefix/name :my.prefix/population]}]}]
#2020-02-2118:30BrianHey y'all I've got a question regarding using transit-js to turn some js into a pathom query. I think this is more transit-based but perhaps someone can identify my issue. My issue has to do with the ' character and () characters in the query:
[{'([:num1 1] {:pathom/context {:num2 1 :num3 1}})
                                         [:sum]}]
because I am using multiple inputs, I'm using a query like this which I've only gotten to work with that ' character and the following ( character which encompasses most of the rest of the query. On the javascript side I'm having a lot of problem turning that into transit because javascript expects different things from the parentheses characters.
transit.map([
      [(transit.keyword("num1"), 1], {
        // pathom/context part should go here, but the parenthesis screws it all up
      }),
      [transit.keyword("ip-blacklisted?")]
    ])
So 1) does anyone know how I can recreate that query using transit of any language implementation (preferably js of course)? Or 2) does anyone know how I can rewrite the query so that it's less convoluted and therefore easier for me to convert with transit?
#2020-02-2119:09myguidingstar@brian.rogers you only need ' in clojure code, not edn strings#2020-02-2119:32myguidingstarAnd the () characters denote a list. Use transit.list to produce one#2020-02-2409:38roklenarcicwhat’s the best way to debug why parser returns not found?#2020-02-2413:19roklenarcicI am trying to bridge two apis in connect, with this being the bridge: given a :country/id you can get to :eb.?????? attributes via such a query:
{'(:eb/countries {:where {:alpha2Code {:eq "GB"}}
                                 :limit 1}) [:eb.??????]}
This query generates the correct graphql call and parse and I’d like to link it to the country attributes. This seems like I need to craft some sort of custom resolver that recursively calls parser with a ā€œfixedā€ AST. What’s the best practice on something like this?
#2020-02-2609:03ouvasamHi, Why do i have "Key must be an Integer" error when doing something like :
(<!! (parser {} [{([:catalogue/id 1] {:where {:a 1}})
                   [:catalogue/name
                    :catalogue/status]}])))
#2020-02-2609:22ouvasamusing this :
(<!! (parser {} [{'(:catalogues {:where {:a 1}})
                   [:catalogue/name
                    :catalogue/status]}])))
I can get the params with (`get-in env [:ast :params])` but when using it with
(<!! (parser {} [{'([:catalogue/id 1] {:where {:a 1}})
                   [:catalogue/name
                    :catalogue/status]}])))
#2020-02-2609:22ouvasami can't get the params ?#2020-02-2609:23ouvasamI also don't understand why i should quote the query ?#2020-02-2610:20ouvasamI found that pathom/context work with that second case as this
(<!! (parser {} [{'([:catalogue/id 1] {:pathom/context {:where {:a 1}}})
                   [:catalogue/name
                    :catalogue/status]}])))
I can have params with (p/entity env) bu this does not work with the first example
(<!! (parser {} [{'(:catalogues {:pathom/context {:where {:a 1}}})
                   [:catalogue/id
                    :catalogue/name]}]))) 
#2020-02-2611:20sifMaybe if you are using fulcro template there is plugin that moves params under query-params in env#2020-02-2612:06ouvasamThanks @U0MK9B06T i can get params with (:com.wsscode.pathom.core/root-query env) but i wonder if it 's the good way to access it ? I don't use fulcro no#2020-02-2610:20ouvasamWhat do i miss here ?#2020-02-2618:01eoliphantHi I was just curious about how folks are organizing their namespaces to best take advantage of prefixing and what have you. as one of my codebases has grown, we’ve taken breaking it up functionally,and making it more or less ā€˜clean’ with something like domain.concept.core (cljc) domain.concept.resolvers (clj[s]) etc,. we’d just been sticking specs for say :domain.concept/foo in the core ns, but recently added an empty domain.concept ns so that we can ::dc/foo and what have you. it seems more convenient, but just feels weird having the empty ns around just for the sake of naming.#2020-02-2620:14Chris O’DonnellI've adopted the same pattern, though I also include spec and coercion logic in the domain.concept.core namespace.#2020-02-2704:01mavbozowe also have empty ns around just for naming attributes for domain specific data. e.g. net.zenius.content so that we can use shorthand ::content/name , ::content/id etc#2020-02-2709:41ak-coramAll the namespaces I use with pathom are empty, so it doesn't feel weird to me. I create them with create-ns and have a macro for defining a consistent set of aliases for them.#2020-02-2713:58eoliphantThanks for the responses, and totally forgot about create-ns.#2020-02-2717:58yendaIn the docs I can see how to send queries to graphql without fulcro but I don't understand what facilities pathom offers for parsing the graphql response#2020-02-2718:02wilkerlucio@yenda welcome šŸ™‚ can you tell more about what you mean on parsing graphql response?#2020-02-2718:10yendaSo I have a re-frame app and I would like to use pathom without fulcro, since I already have a graphql backend I started with the front end, using pathom to query the local state. Now I'd like to explore using it to manage it as well. Currently I have graphql queries as string that I send with axios and denormalize the data "manually". I am playing with query->graphql and it seems to work fine with some tuning to query the endpoint, and I'm wondering if there is an equivalent graphql->query to convert the response to EQL.#2020-02-2718:16wilkerlucio@yenda gotcha, yeah, the query->graphql is something pathom uses internally, I should rename this page here: https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/graphql/fulcro.html#2020-02-2718:16wilkerlucioaltough its named fulcro, this works for anything (just ignore the fulcro remote and UI parts, all the rest should be same, the parser configuration is what matters to you here)#2020-02-2718:18wilkerlucioif you go with the connect part, you can even extend your GraphQL things on the pathom side (created dynamic computations on top of what graphql provides)#2020-02-2718:41yendaOk I'm exploring this, I think that my biggest pain point is to figure how to fill the ident-map, because the endpoint uses attributes like "user_id" "user_name" which translates to "user/id", "user/name" in the app#2020-02-2718:42yendaI had to do the following to generate a valid query:
(defn js-name [k]
  (if (qualified-keyword? k)
    (str (namespace k) "_" (-> k
                               name
                               pg/camel-case))
    k))

(defn ident-transform [[key value]]
  (let [fields [(str (namespace key) "_" (-> key
                                             name
                                             pg/camel-case))]
        value  (if (vector? value) value [value])]
    (if-not (= (count fields) (count value))
      (throw (ex-info "The number of fields on value needs to match the entries" {:key key :value value})))
    {::pg/selector (-> (namespace key) (clojure.string/split #"\.") last)
     ::pg/params   (zipmap fields value)}))

#_(println (pg/query->graphql [{[:user/id "5503438a-583f-11ea-99f8-0242c0a82002"]
                                [:user/id :user/name :user/photo  :user/is-current-user-following]}]
                              {::pg/ident-transform
                               ident-transform ::pg/js-name js-name}))
#2020-02-2718:42yendanot sure how to get the same with the parser#2020-02-2719:21yendaThis is what I'm trying atm:
(def remote-gql
  {::pcg/url       ""
   ::pcg/prefix    "remote"
   ::pcg/mung      clj->gql
   ::pcg/demung    gql->clj
   ::pcg/ident-map {"user"       {"user_id" :remote.user/id}}
   ::p.http/driver request-async})

(take! (pcg/load-index remote-gql) #(reset! indexes %))

(def remote-parser
  (p/parallel-parser
   {::p/env     {::p/reader               [p/map-reader
                                           pc/parallel-reader
                                           pc/open-ident-reader
                                           p/env-placeholder-reader]
                 ::p/placeholder-prefixes #{">"}
                 ::p.http/driver          request-async}
    ::p/mutate  pc/mutate-async
    ::p/plugins [(pc/connect-plugin {; we can specify the index for the connect plugin to use
                                        ; instead of creating a new one internally
                                     ::pc/indexes  indexes})
                 p/error-handler-plugin
                 p/trace-plugin]}))

(take! (remote-parser {} [{[:remote.user/id "5503438a-583f-11ea-99f8-0242c0a82002"]
                           [:remote.user/id :remote.user/name :user/photo]}]) println)
#2020-02-2719:21yendaand I get {[:remote.user/id 5503438a-583f-11ea-99f8-0242c0a82002] {:remote.user/id 5503438a-583f-11ea-99f8-0242c0a82002, :user/photo :com.wsscode.pathom.core/not-found, :remote.user/name :com.wsscode.pathom.core/not-found}}#2020-02-2719:22yendarequest-async is just the regular one but I added a print to see which request is sent, seems like it doesn't send any#2020-02-2719:37yendaI can see the indexes loaded properly#2020-02-2719:40yendamhm okay so I get a result with
(take! (remote-parser {} [{[:remote.User/user_id "5503438a-583f-11ea-99f8-0242c0a82002"]
                           [:remote.User/user_id :remote.User/user_name :remote.User/user_photo]}]) println)
#2020-02-2719:55yendais there a way to add some headers to the http driver?#2020-02-2720:38yendaok as simple as passing ::http/headers to the env of the parser#2020-02-2720:40yendais there a way to "fallback" on the remote parser when the local one doesn't find something?#2020-02-2806:57Jakub Holý (HolyJak)FYI there is err in https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/resolvers.html#_single_attribute_resolvers - the text reads "use single-attr-resolver2" but the code below (pc/single-attr-resolver ::some-value ::other-value single-with-env))#2020-02-2815:16wilkerluciothanks, fixed on master, gonna deploy updated docs later#2020-02-2815:23yendaI have a defresolver that already returns some of the nested attributes after running a big composed sql query with json aggregates, how can I query these nested attributes? I get :com.wsscode.pathom.core/not-found while if I don't ask for any and just put the key I get the map that is there in the reply#2020-02-2815:26yendainterstingly if I use placeholder syntax I get the queried attributes, but only because another resolver is used for that#2020-02-2815:27yendaIn graphql I had
(defn- idempotent [field next] (fn [root args context info]
    (if-let [field-value (get (utils/gql->clj root) field)]
      field-value
      (next root args context info))))
around resolvers for attributes that might have already been solved by earlier resolvers
#2020-02-2815:28yendaWhat would be the best approach with pathom? So far I'm pretty stoked by the lib and looking forward to replace graphql entirely#2020-02-2817:01yendaok it was somewhat easier than I thought, I simply add to nest the subfields in the ::pc/output#2020-02-2817:02yendaI guess the doc could show an example of that, I was hinted by https://wilkerlucio.github.io/pathom/#_query_notation_introduction#2020-02-2817:04yendaactually the reason I couldn't get it to work was slightly more complex, I was reading :query in the ast to get the fields and compose the sql query and naively expected keywords, but ofc as soon as I started specifying subfields, the elements in the query also contained maps, so my sql query didn't contain the field I was looking for anymore šŸ˜„#2020-03-0218:04tvaughanSorry if this has been answered before, or if I missed this somewhere in the documentation. I'm working with Fulcro and Datomic. I worked with Om.Next way back when and implemented my own server-side API. This API allowed queries like: [:foobar/by-ident any-ident-with-a-unique-constraint] such as [:foobar/by-ident " or [:foobar/by-ident 123456789] where {:tag :mailto:someoneexample.comsomeoneexample.com, :attrs nil, :content nil} could be the email address of a person entity with a unique constraint, 123456789 could be the tax id number of a company entity with a unique constraint, and :foobar is just an arbitrary namespace name. I think I see how I could implement something similar with Fulcro and Pathom, however I'm confused by the use of output on defresolver. I understand the desire to whitelist return values, but let's assume I have another way to limit what can be returned. Can I simply omit output if I want to allow the resolver to return anything? Like this?
(defn q-by-ident
  [db ident & [query]]
  (datomic/pull db (or query '[*]) ident))

(pc/defresolver by-ident-resolver
  [env {:foobar/keys [ident]}]
  {::connect/input #{:foobar/by-ident}}
  (q-by-ident ident))
#2020-03-0218:06souenzzo@tvaughan no connect always need a output It will only call a resolver when the query "requires" a key that a resolver can output#2020-03-0218:07tvaughanI see. Thank you for the quick response @souenzzo#2020-03-0218:10souenzzo@tvaughan your resolver don't need to always output all keys from ::pc/output The meaning of pc/output is "this resolver may return this keys" If the resolver don't return, it will try "the next" resolver that provides that key#2020-03-0218:11tvaughanUnderstood. Thanks for the clarification @souenzzo#2020-03-0218:22wilkerlucio@tvaughan another thiing to notice is that Pathom changed the standard pattern of by-*, instead I recommend the usage the property directly (instead of :foobar/by-id, use :foobar/id), this is part of a greater vision Pathom has around property re-use, this way the id natually gets to be a hub, and reduce unescessary naming convertions between by-id and id around namespaces#2020-03-0218:34tvaughanby-ident is purely an invention of mine that only exists between the client and server. by-ident doesn't exist anywhere in a datomic schema. I also have a by-attrs that can be used to return a list of entities according to some attribute value that doesn't have a unique constraint, like a last name. I haven't come across anything yet in the Pathom documentation about id and property re-use. I'll take another look. Thanks @wilkerlucio#2020-03-0219:15wilkerlucioI understand, just pointing it out because by-id was a standard convention in om.next times, but not anymore šŸ™‚#2020-03-0223:05mdhaneyHow do I use request-caching with connect? Or maybe a better question is, how do I access the inputs being passed to a connect resolver given just the ā€˜env’, since it looks like the cached functions are expecting a single ā€˜env’ argument?#2020-03-0223:21mdhaneySome context for what I’m trying to do: I’m using Datomic Cloud and my app is setup to run as an Ion, or locally for development (in which case all the queries are remote calls). On startup, my app issues a very large query that pulls in most of the graph for the logged-in user. This works fine in production, but consistently hangs during parsing when I run locally. The problem seems to be the huge number of queries being issued to Datomic, which are all now remote calls and apparently are overwhelming the access gateway. I’ve temporarily corrected this by adding a special resolver for this one query, which pulls as much data as it can in a single call to Datomic. I an pull in maybe 70% of the graph this way, but additional calls are still needed and in fact this solution didn’t even work until I highly optimized a couple of the additional queries. So I’m worried it will break again at some point when new queries are added, and of course when the local development version is locking up it seriously impacts productivity. Plus, having to tailor to specific queries like this loses a lot of the flexibility pathom gives me. Since there is a lot of redundant information in the graph, I think request caching will help here so that I can keep my more granular resolvers, but not have the database being queried multiple times for the same data.#2020-03-0300:45yendanot sure if that answers your question but you can enrich the env by simply returning an additional ::p/env key in your resolver, which would be (assoc env :your-stuff its-value)#2020-03-0300:47yendaalso did you check https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/core/request-cache.html ?#2020-03-0300:56yendait looks like the cached functions are expecting a single 'env' argument? are you sure about that? in the example there is just one env argument but that doesn't mean you can't make your own, as explained in the comment you can have additional arguments like id for instance and make it part of the key, so that the cache can work with multiple queries#2020-03-0301:19wilkerlucio@U0JPBB10W thanks for the detailed explanation, here is an example of how you can create a persistent caching mechanism: https://gist.github.com/wilkerlucio/a096df18f65fbbf57f62cf96bc168e8d#2020-03-0301:19wilkerluciowhat is really missing there is how to expire it, you have to take care of that, maybe by time, maybe by something manually cleaning the cache, whatever works better on your system#2020-03-0301:26mdhaneyThanks, I’ll take a look at that. I don’t really need a persistent cache - just caching while processing a single query should be enough. But it’s nice to have another example, to make sure I’m understanding the request-cache correctly.#2020-03-0302:58mdhaneyThanks @wilkerlucio , that transform code is exactly what I needed! Now I’m able to use more granular resolvers, and it’s only making 2/3 as many DB queries as the version with the specialized resolver. Now I’m making modifications to support batching and still using the caching effectively. Once that’s in place, it should cut the number of queries by half at least!#2020-03-0312:29yendaSorry to hijack the thread but I have a cache related question as well: would a per request-cache be useful in a parallel reader if the queries are at the same "level"? i.e if I query a list of messages and the user details are filled by another resolver, wouldn't it run the same query for identical users when the parallel reader goes through them#2020-03-0313:44wilkerlucio@U0DB715GU it can be, if you are the same level, you would probably hit the entity cache first, since the data is already there, but depending on the processing, maybe a resolver got scheduled to run for multiple reasons (different attributes, that have differnet goals, but share things in the middle), in this case the request cache kicks in to avoid duplicated calls to the resolver. another way it can be used is if you are dealing with farther appart parts of the same requests that end up needing similar things#2020-03-0301:19wilkerlucio#2020-03-0413:45tvaughanLet's say I have a query like [{[:person/id " How do I get access to [:person/name] in my resolver? I have a resolver like:
(pc/defresolver person-resolver
  [env {:keys [person/id] :as params}]
  {::pc/input #{:person/id}
   ::pc/output [:person/name :person/email]}
...)
and I don't see where I could grab this out of env
#2020-03-0413:52wilkerlucio@tvaughan in the resolver you described, you have to provide the :person/name, if you want to use :person/name as input for something, you can create another resolver that depends on it, makes sense?#2020-03-0413:56tvaughan> makes sense? No, sorry. I mean a client can submit a query like: [[:person/id 42]] which means "return everything" or [{[:person/id 42][:person/name]}] which means "return only :person/name. Right now the resolver returns everything and Pathom returns only what has been requested. However, in my resolver I don't want to fetch everything, I only want to fetch what the client requested. Which means my resolver needs to know what the client sent beyond just the person id. How?#2020-03-0413:57yendain env you want to get ::p/keys [parent-query]#2020-03-0413:58yendafor instance I do
(reduce (fn [acc field]
                          (conj acc (if (keyword? field)
                                      field
                                      (first (first field)))))
                        #{}
                        parent-query)
to then only select those fields in my sql query
#2020-03-0413:59yendaif the query is a mutation then I do (get-in env [ast :query])#2020-03-0414:12tvaughanBingo! Thanks @yenda#2020-03-0414:24souenzzoReturn basead on parent-query isn't always a good solution It may kill the "connect superpowers"
(let [register [(pc/resolver
                  `user-by-id
                  {::pc/input  #{:user/id}
                   ::pc/output [:user/email
                                :user/name]}
                  (fn [{::p/keys [parent-query]} {:user/keys [id]}]
                    (-> {:user/email (str id "@example.com")
                         :user/name  (str id)}
                        (p/map-select parent-query))))
                (pc/resolver
                  `slug-by-email
                  {::pc/input  #{:user/email}
                   ::pc/output [:user/slug]}
                  (fn [env {:user/keys [email]}]
                    {:user/slug (first (string/split email #"@"))}))]
      parser (p/parser {::p/plugins [(pc/connect-plugin {::pc/register register})]})
      env {::p/reader              [p/map-reader
                                    pc/reader2
                                    pc/open-ident-reader]}]
  (parser env `[{[:user/id 42] [:user/name
                                :user/slug
                                :user/id
                                #_:user/email]}]))
#2020-03-0415:03tvaughanThanks @U2J4FRT2T This is what I have now:
(pc/defresolver person-resolver
  [{:keys [::p/parent-query] :as env} {:keys [person/id] :as params}]
  {::pc/input #{:person/id}
   ::pc/output [:person/name :person/email]}
  (datomic/q-by-ident (datomic/conn->db conn) [:person/email id] parent-query))
#2020-03-0415:07souenzzoI already used it in a project and I moved back to (d/pull db (-> env ::pc/resolver-data ::pc/output) ei ) after some connect issues#2020-03-0415:08souenzzosee also: https://github.com/souenzzo/eql-datomic/#2020-03-0415:28tvaughanNice!#2020-03-0416:03yendaI dont understand could you elaborate what it kills?#2020-03-0416:04yendaWhat is the alternative?#2020-03-0416:29souenzzoalways return "everything" in resolver#2020-03-0419:24yendafor me that is not an option the resolver is doing a composed sql query with different aggregates to limit roundtrips to the db server#2020-03-0414:15tvaughanOh, it seems as though Pathom doesn't support a query like my first example: [[:person/id 42]], right? Instead Pathom requires queries like my second example: [{[:person/id 42][:person/name :person/preferred-pronouns]}] , correct? A query like [[:person/id 42]] doesn't trigger the person/id resolver#2020-03-0414:16wilkerlucio@tvaughan exactly, pathom only loads things that the user requires, you may have dozens of resolvers that can get something from :person/id, Pathom optimizes to fulfill the client demand (which should be precise, the EQL request must be explicit about what it wants, by design)#2020-03-0417:21mssis there an easy way when getting a parallel read timeout w/ the async parser to trace down what resolver is timing out?#2020-03-0418:21kszabonot by default AFAIK#2020-03-0513:00yendais there a recommended pattern to fallback on a remote parser when a local parser didn't find all keys? should it be a mega resolver or a last reader?#2020-03-0513:11wilkerlucio@yenda that's whats coming on the next version šŸ™‚ I have two projects using a setup in the way you described, they have both local parsers (running on CLJS) and remote parsers (running on some server), for basic cases it already works, but I still have to document a lot of it, and I'm still working on some tooling, so until I have docs and tools I dont recommend it for general usage. but if you are feeling adventurous, there are examples in the tests: https://github.com/wilkerlucio/pathom/blob/query-planner/test/com/wsscode/pathom/connect/planner_readers_test.cljc#L328-L562 (use the alpha releases to have it available)#2020-03-0513:12wilkerlucioI also like to improve the visibility of the work I'm doing, because that's been a quite long work process to get this out, and I like to provide something so people that want to follow can know whats going on, not going deep on this now, but expect some news around that soon šŸ™‚#2020-03-0513:25yenda@wilkerlucio that look awesome! In my case I have a pretty simple use case because I only have one remote and I know that whatever isn't solved locally will be solved remotely, so I was basically looking for a solution to send the remainers of the query that have not been solved to the remote#2020-03-0513:25yendabut maybe it isn't that simple, I was hoping there would be a way to get the remaining query after the local parser ran and just send it to the remote parser#2020-03-0513:27wilkerluciothis can be done with some map manipulation code, get the query, query the data, make a diff and run, I guess its a simpler problem#2020-03-0513:27yendaand were would you do that? in a custom reader?#2020-03-0513:29wilkerluciodepends on the case, are you doing it in a fulcro app?#2020-03-0513:30yendano re-frame#2020-03-0513:33yendaI run the local parser on app-db#2020-03-0513:34yendaI'd like to know what is the best place to rebuild a query based on all the ::p/not-found in the result of the local parser#2020-03-0513:37yendamaybe just work it out from the result of the local parser and the initial query#2020-03-0513:46yendaas a first step I could just re-run the whole query, I'd just need a way to know if the response is incomplete without looking at it#2020-03-0614:51yendaSo if someone ends up having the same idea what I did is using a raw subscription:
(defn subscribe
  [local-query]
  (re-frame/subscribe [::query local-query]))

(re-frame/reg-sub-raw
 ::query
 (fn [app-db [_ [query-subscription-key & args :as query-subscription]]]
   (let [query-fn (query-subscription-key->query-fn query-subscription-key)
         query [(apply query-fn args)]]
     (take! (eql.local-parser/parser {:app-db app-db} query)
            (fn [response]
              (re-frame/dispatch [::local-response query-subscription query response])))
     (reagent.ratom/make-reaction
      (fn [] (get-in @app-db [::queries query-subscription]))
      :on-dispose #(re-frame/dispatch [::query-unsubscribed query-subscription])))))
then I cleanup the response
(clojure.walk/prewalk
                          (fn [v]
                            (if (and (keyword? v) (= v ::p/not-found))
                              nil
                              v))
                          response)
if the cleaned-up response is the same as the response I don't do a remote query However I decided to give up on it because I realized it was pointless, I wouldn't even get proper updates of the subscription when doing a mutation that changes the data for instance. Since the app-db is already denormalized by being filled upon reception of a remote response, I decided to simply work locally with regular re-frame susbcription and skip the local parser completly. So it works like that: • I make remote queries as soon as I know I need the data (for instance when app loads, big ass query, when re-frame event navigating to a screen is dispatched, query the data that will be needed for that screen, etc...) • When I receive a response, I have a reduce-handler function that goes over each key of the response map and calls a defmulti handler that dispatches on the key, which in some keys is called recursively if results are nested. These handlers are doing what has to be done in the app based on the results, such as populated app-db in a normalized way but also sometime navigating or doing other effects • Components use re-frame subscriptions to get the data, it doesn't try to denormalize the data, so if a component needs a person, it susbcribes to [:person person-id], if a sub component needs the list of contacts, person has a vector of contact-ids, so to show a list of contacts component, it would be passed the vector of ids, and each element of the list would subscribe to [:person contact-id] That way I can keep app-db normalized, and I don't have to make local queries, when something changes in the db as a result of a mutation or new query, only the concerned components will update. Ofc there might be some inefficiencies, some data may be queried more often than necessary. I will look into it in the future as it is not a big problem for now, especially since the app-db is basically a cache and some data would need a cache invalidation strategy so until bandwith becomes an issue it's fine. A potential solution would be to add a key to entities I don't want to requery and check that before sending a remote query
#2020-03-0614:55kszaboYou can replace your prewalk with this plugin: https://github.com/wilkerlucio/pathom/commit/fed5f103dcf5a63df270e02b05d0429e50595dad#2020-03-0614:57yendathanks for the tip! how would I know that the parser didn't find everything I want though? Anyway as I said I gave up on the local parser as I realized it was adding unnecessary complexity and it was adding some issues as well#2020-03-0614:59yendamy main goal was to see if I could have a simple easy way to only query the missing data on the remote parser, but for that I am not sure having a local parser is the best strategy since re-frame susbcriptions are much simpler and faster to display data, and I could find something else on the remote query side#2020-03-1014:54souenzzoHello I'm trying to use datomic-connect-plugin But my parser just returns the data inserted after it's creation (it kind of snapshot my DB at de creation of the parser)
com.wsscode/pathom-datomic {:git/url ""
                            :sha     "23e23ac8ae96dba5461cd020879d896c355f65d3"}
_
(let [parser-from-conn (fn [conn]
                         (p/parser {::p/env     {::p/placeholder-prefixes #{">"}
                                                 ::p/reader               [p/map-reader
                                                                           pc/reader2
                                                                           pc/open-ident-reader
                                                                           p/env-placeholder-reader]}
                                    ::p/mutate  pc/mutate
                                    ::p/plugins [(pc/connect-plugin {::pc/register pcd/registry})
                                                 (pcd/datomic-connect-plugin (assoc on-prem-config ::pcd/conn conn))
                                                 p/error-handler-plugin
                                                 p/trace-plugin]}))
      db-uri (doto (str "datomic:mem://" (UUID/randomUUID))
               d/create-database)
      tx-schema [{:db/ident       :user/id
                  :db/unique      :db.unique/identity
                  :db/valueType   :db.type/string
                  :db/cardinality :db.cardinality/one}
                 {:db/ident       :user/name
                  :db/valueType   :db.type/string
                  :db/cardinality :db.cardinality/one}]
      conn (d/connect db-uri)
      _ @(d/transact conn tx-schema)
      p1 (parser-from-conn conn)
      tx-data [{:user/id   "1"
                :user/name "one"}]
      {:keys [db-after]} @(d/transact conn tx-data)
      p2 (parser-from-conn conn)
      parser (juxt p1 p2)
      query [{[:user/id "1"] [:user/name]}]]
  {:without-db    (parser {::pcd/conn conn
                           ::p/entity {::pcd/conn conn}}
                          query)
   :with-db-after (parser {::pcd/conn conn
                           ::pcd/db   db-after
                           ::p/entity {::pcd/conn conn
                                       ::pcd/db   db-after}}
                          query)})
=>
{:without-db [{[:user/id "1"] {:user/name :com.wsscode.pathom.core/not-found}} {[:user/id "1"] {:user/name "one"}}],
 :with-db-after [{[:user/id "1"] {:user/name :com.wsscode.pathom.core/not-found}} {[:user/id "1"] {:user/name "one"}}]}
#2020-03-1021:47souenzzo@U066U8JQJ#2020-03-1113:21wilkerlucioyeah, the DB is kind static on that sense, I have not got to this point, but I think its an important one to tackle, can you open an issue on pathom-datomic to keep track of it?#2020-03-1109:03kHi. I’m trying to integrate GraphQL with pathom, and translate GraphQL queries into EQL queries. But don’t figure out the EQL format for this GraphQL query like follows:
mutation {
  logout { # no `()` after `logout`
    success
  }
}
Any help is appreciated.
#2020-03-1109:55souenzzoIm working on it here https://github.com/souenzzo/eql-gql/blob/master/src/test/br/com/souenzzo/eql_gql_test.clj#L183#2020-03-1111:30souenzzoLet's move this topic to #eql channel#2020-03-1111:39kThanks @U2J4FRT2T, I think it may be relevant to this pathom module: com.wsscode.pathom.graphql/query->graphql#2020-03-1111:45souenzzoAnyway this mutation in EQL should be something like
[{(app.user/logout {})
  [:app.status/success]}]
#2020-03-1211:52Bjƶrn Ebbinghaus
com.wsscode.pathom.fulcro.network
is depending on fulcro 2 Is there a way to use pathom on the client with fulcro 3?
#2020-03-1217:43wilkerlucio@mroerni yes, but you have to implement, its quite simple, here is an example:
(ns pathom-remote-fulcro3
  (:require [com.fulcrologic.fulcro.algorithms.tx-processing :as txn]
            [com.wsscode.async.async-cljs :refer [go-promise <?maybe]]
            [edn-query-language.core :as eql]))

(defn pathom-remote [parser]
  {:transmit!
   (fn transmit! [_ {::txn/keys [ast result-handler]}]
     (let [edn           (eql/ast->query ast)
           ok-handler    (fn [result]
                           (try
                             (result-handler (assoc result :status-code 200))
                             (catch :default e
                               (js/console.error e "Result handler for remote failed with an exception."))))
           error-handler (fn [error-result]
                           (try
                             (result-handler (merge error-result {:status-code 500}))
                             (catch :default e
                               (js/console.error e "Error handler for remote failed with an exception."))))]
       (go-promise
         (try
           (ok-handler {:body (<?maybe (parser {} edn))})
           (catch :default e
             (js/console.error "Pathom Remote error:" e)
             (error-handler {:body e}))))))})
#2020-03-1218:56myguidingstar@wilkerlucio a dynamic resolver is one that doesn't have fixed input and output. What does ::pc/provides of such resolver look like? What about oir and io indexes for dynamic resolvers?#2020-03-1219:26wilkerluciothere is no output or provides on dynamic resolvers (in the new model) in the past there was a ::pc/compute-output, but that's not used in the new reader. what matters is how you build the index to point to this resolvers, Pathom will try to accumulate all the calls for that and run a single one#2020-03-1219:26wilkerluciothere is some things to figure still, if you look at how pathom is dealing with foreign parsers, its another example of how dynamic resolvers work#2020-03-1219:27wilkerluciohttps://github.com/wilkerlucio/pathom/blob/query-planner/test/com/wsscode/pathom/connect/planner_readers_test.cljc#L333#2020-03-1219:00myguidingstaranother question: why the code here throw exception if ::pc/output is not specified? https://gist.github.com/myguidingstar/06d912afb73087bba6b3a01abf7e9db8 (Though the pathom-datomic resolver still doesn't have ::pc/output!!!)#2020-03-1219:26wilkerluciowhat exactly throws that? is this some spec validation? it should not require it#2020-03-1409:50myguidingstar{:a/x :com.wsscode.pathom.core/reader-error, :com.wsscode.pathom.core/errors {[:a/x] "class clojure.lang.ExceptionInfo: No output available - #:com.wsscode.pathom.connect{:sym dev/my-resolver}"}}#2020-03-1409:51myguidingstarthe result if I comment out the line ::pc/output []#2020-03-1221:01otwieraczHey, where can I find any information how to build Pathom API on top of Ring?#2020-03-1221:01otwieracz(just like with Fulcro, but I'd like to try using it without it)#2020-03-1221:53wilkerlucio@slawek098 the parser is just a function, in any case the work is about reading the EQL query somehow (usually on the POST body), pass that to the parser and encode the response#2020-03-1221:55otwieraczYes, I started to wrap it on my own.#2020-03-1221:55otwieraczSeems like it's working.#2020-03-1221:55otwieraczShould be at least. Now I need to ask some transit queries from frontend.#2020-03-1222:42sheluchinI'm trying to better understand how to use Walkable. Do I still need to explicitly define resolvers which wrap parser calls, or is Walkable abstracting that away?#2020-03-1314:00myguidingstar@alex.sheluchin for now you still need to set up parser like in walkable's documentation. Walkable was created before pathom connect. With the new dynamic resolver feature coming in pathom 2.3.0-alpha4, integration with pathom connect is possible - I'll try to make connect version of walkable soon, hopefully next week#2020-03-1323:06sheluchin@U0E2YV1UZ thank you. I think my use case is simple enough to stick with what's available, I was just trying to understand the correct pattern to use. I'll be looking forward to making use of the Connect support once that does come around.#2020-03-1317:05otwieraczI've got a case where :note/objects can have :text-object/text property or :tag-link/tag property: :note/objectsĀ [:text-object/textĀ :tag-link/tag]} but with this syntax I am getting a lot of :com.wsscode.pathom.core/not-found#2020-03-1317:05otwieraczWhat's the correct approach here?#2020-03-1317:13kszabohttps://github.com/wilkerlucio/pathom/commit/fed5f103dcf5a63df270e02b05d0429e50595dad remove those with this plugin šŸ™‚#2020-03-1317:13kszaboit’s perfectly fine for entities to have fewer attributes than requested#2020-03-1317:15kszaboalternatively if these represent different entities altogether, then use union queries: https://wilkerlucio.github.io/pathom/#_union_queries#2020-03-1318:13otwieraczOh, unions might be great!#2020-03-1318:45otwieracz
::p/plugins [(pc/connect-plugin {::pc/register [(resolvers/resolvers)
                                                             (mutations/mutations)]})
                          p/elide-special-outputs-plugin
                          p/error-handler-plugin
                          ]}))
#2020-03-1318:45otwieraczJust like that, and this should get rid of any :com.wsscode.pathom.core/not-found?#2020-03-1318:48otwieraczhmm... let me try#2020-03-1318:48otwieracz(once again)#2020-03-1318:48otwieraczMaybe something isn't using new definition#2020-03-1318:51otwieraczYay, it works! Thank you!#2020-03-1319:27otwieraczAre there any facilities to deal with complex EQL queries? When I need to ask for deeply nested, complex structure - it quickly becomes difficult to manage.#2020-03-1320:09yenda@slawek098 you make a queries namespace and you write functions that build your queries#2020-03-1320:10yendaqueries are basically clojure datastructure so nothing easier than that#2020-03-1320:10yendafor the mutations use the list function so you can pass arguments#2020-03-1320:11yenda
(defn login
  [{:user/keys [username email password]}]
  {(list 'login (cond-> {:user/password password}
                  email (assoc :user/email email)
                  username (assoc :user/username username)))
   [{:auth/tokens [:access-token
                   :expires-in
                   :offline-token
                   :id-token]}
    {:auth/user [:user/id
                 :user/username
                 :user/name
                 :user/photo
                 (user-stuff {})]}
    (search-things {:limit 3})]})
#2020-03-1320:13yendafor instance here's a login function that calls the login mutation, but also a bunch of other stuff including parametrized queries to get after the user is logged in (for that I modify the env of the parser and add the authed user after succesful login)#2020-03-1513:38kwladykaJust want to make author(s) of pathom happy šŸ™‚ Thank you man. You are doing really good job. I just started to play with this, but it looks very promising. It has potential to be one of my favorite.#2020-03-1518:39kwladykahttps://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/introduction.html my humble suggestion for the doc: search option in the doc is have to!#2020-03-1713:18wilkerlucioagreed, just didn't had the time to prioritize yet, if you are feeling brave on how to add, I'm happy to take PR's :)#2020-03-1521:07kwladykaDo you prefer to use with pathom plurlar or singular functions? So get-shop and make many queries in request or get-shops and use collection in response?#2020-03-1521:14kszabodepends on the backing datasource, if you can’t write a batch resolver, just write a singular one, but prefer batch for N+1 query reasons#2020-03-1522:33kwladykaI think I can do whatever I want. But the choice is hard šŸ™‚#2020-03-1607:50yendait depends on how much you want to hammer your db, I have resolvers that compose a fat query and return most of the data already to minimize back and forth#2020-03-1612:16kwladyka
(utils/request-eql '[{[:shop/uuid #uuid"00000000-0000-0000-0000-000000000000"] [* :shop/name]}])
return
#:shop{:uuid #uuid"00000000-0000-0000-0000-000000000000",
       :name "redshop",
       :engine "atomstore",
       :config {:foo "bar"},
       :updated_at #inst"2020-03-15T23:03:19.353-00:00",
       :created_at #inst"2020-03-15T23:03:19.353-00:00"}
but
(utils/request-eql '[{[:shop/uuid #uuid"00000000-0000-0000-0000-000000000000"] [*]}])
return
#:shop{:uuid #uuid"00000000-0000-0000-0000-000000000000"}
For resolver:
(pc/defresolver get-shop [env {:shop/keys [uuid]}]
                {::pc/input #{:shop/uuid}
                 ::pc/output [:shop/uuid :shop/name :shop/engine :shop/config]}
                (shop-biz/get-shop uuid))
How to get ALL values? I was trying also
(utils/request-eql '[[:shop/uuid #uuid"00000000-0000-0000-0000-000000000000"]])
#2020-03-1612:16kwladykaIs it a bug?#2020-03-1612:17kwladyka* works if I want any other value like :shop/name but if on it does nothing#2020-03-1612:52kszabo* just returns the current entity#2020-03-1612:52kszaboit just means, any resolvers that got called, I want all the data#2020-03-1612:53kszaboif you just supply * no resolvers will be called, hence you only get what you provided for base entity#2020-03-1612:54kszabothere is no way to say I want all direct outputs (or n-levels deep) from this attribute that I provided#2020-03-1613:24kwladykaHow does it work here https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/resolvers.html#GlobalResolvers ?#2020-03-1613:24kwladykaor here https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/resolvers.html#_resolvers_with_single_input#2020-03-1613:24kwladykafor [{::latest-product [*]}] I got
{::latest-product
 {:product/id 1,
  :product/title "Acoustic Guitar", 
  :product/price 199.99}}
#2020-03-1613:26kwladykaalso output make me confuse because:
::pc/output [:shop/uuid :shop/name :shop/engine :shop/config]}
can return
#:shop{:uuid #uuid"00000000-0000-0000-0000-000000000000",
       :name "redshop",
       :engine "atomstore",
       :config {:foo "bar"},
       :updated_at #inst"2020-03-15T23:03:19.353-00:00",
       :created_at #inst"2020-03-15T23:03:19.353-00:00"}
#2020-03-1613:26kwladykaupdated_at is not in output#2020-03-1613:27kwladykaso this output looks a little like something to ingore#2020-03-1613:27kwladykabesides of maybe informational purpose#2020-03-1613:45kszabo::pc/output is just describing that these things can be returned (there can be more)#2020-03-1613:45kszaboand * works as I described, you are using it inside a join#2020-03-1613:45kszaboanyway that is for exploratory cases and shouldn’t be the way you use Pathom in real code#2020-03-1613:47kwladyka
(pc/defresolver get-shop [env {:shop/keys [uuid]}]
                {::pc/input #{:shop/uuid}
                 ::pc/output [:shop [:shop/uuid :shop/name :shop/engine :shop/config]]}
                {:shop (shop-biz/get-shop uuid)})
(pc/defresolver get-shop [env {:shop/keys [uuid]}]
                {::pc/input #{:shop/uuid}
                 ::pc/output [:shop/uuid :shop/name :shop/engine :shop/config]}
                (shop-biz/get-shop uuid))
Do you see any pros / cons between this 2 ver.? So return data in :shop vs directly
#2020-03-1713:20wilkerlucioas people described, flat is better, but just a hint, the first example is missing the map {} to make the join between :ship and the nested properties#2020-03-1613:48kwladykaat that moment I am mainly focus on testing part, so no write FE side#2020-03-1613:48kwladykaso can not see some issues#2020-03-1613:50kszaboKeep your resolvers flat, reserve nesting for joins (when you need to return multiple maps/etc. like {:user/friends [:user/id]} )#2020-03-1613:59kszabohttps://github.com/wilkerlucio/pathom-connect-youtube/blob/master/src/core/com/wsscode/pathom/connect/youtube/search.cljs#2020-03-1613:59kszabothis could be useful#2020-03-1614:01kwladykahmm after all I think get-shop doesn’t make sense. get-shops is probably what I want.#2020-03-1614:03kwladykaDo you use create-shop / create-shops ? I know this questions sounds dump, but pathom in my feeling gives possibilities of different approaches which can be still simple#2020-03-1614:09eoliphantid say it depends on your use cases. Do you have scenarios where you bulk create ā€œshopsā€#2020-03-1614:09kwladykaprobably not, but I would like to keep the same rule for all queries. plural or singular, but not both.#2020-03-1614:09kwladykaprobably#2020-03-1614:37eoliphantyeah, you might just want to experiment a bit, especially for my mutations, i tend to try to let them reflect the ā€˜commands’ in my domain. nice thing is that if you follow the guidance of doing most of the work in separate functions that your resolvers call, it’s a little easier to come up with composable bits that are easier to leverage in whatever resolver approach you settle on#2020-03-1614:55kwladykaafter all I think I will do get-items but create update delete 1 item#2020-03-1614:56kwladykaJust because managing state is harder and debugging 1 create is easier, than many#2020-03-1615:07souenzzoIn my experience, there is no create operation in any system create-user usually will create user, session, message buy-item will create cart itemInCart ... that CRUD thing simply don't exists nor help#2020-03-1614:56eoliphantanother thing i’ve found is that resolver names are themselves less of a big deal, for instance, i’ve got some code that I use to auto-generate some of them, the real deal is what they’re contributing to the resolver graph.#2020-03-1614:57eoliphantsure, go with that, and see how it works out for you.. and like i said, if your resolvers and mutations are delegating most of their work to helpers, say a create-many-items shouldn’t be a big lift if you end up needing it#2020-03-1615:01eoliphanti’ve had situations where pretty much all of the ā€˜app driven’ stuff was more or less your model, create-one but later i needed some bulk import capability. So just whipped up some code that used my create-one helper functions for a bulk mport process that was run by ops in the background#2020-03-1615:01kwladykaBTW what is your pattern to get all items vs selected items?
(utils/request-eql [{[:shop/uuids #{}] [:shops]}])
I am thinking about send empty uuids as ā€œget allā€
#2020-03-1615:02eoliphanti use explicit {:all-xxx …} resolvers for now#2020-03-1615:02kwladykawhat pros?#2020-03-1615:04souenzzo@kwladyka take a look at https://www.infoq.com/presentations/domain-driven-architecture/#2020-03-1615:04kwladyka
(utils/request-eql [{[:shop/uuids :all] [:shops]}])
I can also use :all or whatever
#2020-03-1615:04eoliphantin my case, i’m using datomic on the backend, and this little bit of meta/domain modeling code i wrote, so those resolvers by and large just get generated. I’ve found that the cases where i need one v many tend to not really overlap too much so by applying consistently i’m generally ok.#2020-03-1615:06eoliphantyeah that’s the beauty of it, you can do almost whatever you want. though your mileage will of course vary. One thing you might want to play around with in your approach is passing whatever params you need (paging, limits, whatever).#2020-03-1615:08kwladykaI changed this to :all for now, but I will see#2020-03-1615:25kwladyka
(let [shop #:shop{:uuid #uuid"11111111-1111-1111-1111-111111111111"
                  :name "test shop"
                  :engine "engine"
                  :config "{\"abc\": \"def\"}"}]
  (is (= shop
         (utils/request-eql [{'(shop/update shop)
                              [:shop/uuid :shop/name :shop/engine :shop/config]}]))
      "update shop"))
How do you achieve
'(shop/update shop)
I mean I want to call mutation, but I want to have shop from let
#2020-03-1615:26kwladykaI use pr-str to conver this into string
#2020-03-1615:27kwladykaor do you know better method to generate eql query?#2020-03-1615:29kwladykaI can always do
(list 'shop/update shop)
but this is ugly
#2020-03-1615:29kwladykahmm but probably have to#2020-03-1615:33kwladykaoh I can use
`(shop/update ~shop)
#2020-03-1616:26souenzzopr-str i think that is the default method to generate a string from edn but usually we use transit once it runs on browser#2020-03-1620:35kwladykaquery
(let [shop #:shop{:uuid #uuid"11111111-1111-1111-1111-111111111111"
                  :name "test shop"
                  :engine "engine"
                  :config {:abc "def"}}]
  (utils/request-eql [{`(shop/update ~shop) [:shops [:shop/uuid :shop/name :shop/engine :shop/config]]}]))
response
#:shop{update
       {[:shop/uuid :shop/name :shop/engine :shop/config]
        #:shop{:uuid :shop/name},
        :shops
        [#:shop{:uuid #uuid "11111111-1111-1111-1111-111111111111",
                :name "test shop",
                :engine "engine",
                :config {:abc "def"},
                :updated_at #inst "2020-03-16T20:33:21.279-00:00",
                :created_at #inst "2020-03-16T20:18:03.964-00:00"}]}} 
^ what #:shop{:uuid :shop/name} doing here? defs
(pc/defmutation update-shop [env params]
                {::pc/sym 'shop/update
                 ::pc/params #{:shop/uuid :shop/name :shop/engine :shop/config}
                 ::pc/output [:update]}
                (shop-biz/update-shop params)
                {:shop/uuids #{"11111111-1111-1111-1111-111111111111"}})

(pc/defresolver get-shops [env {:shop/keys [uuids]}]
                {::pc/input #{:shop/uuids}
                 ::pc/output [:shops [:shop/uuid :shop/engine]]}
                {:shops (shop-biz/get-shops uuids)})
#2020-03-1620:38kwladykaHow to code this correctly? So update-shop return shop with get-shops#2020-03-1621:16kwladykaI have to say after all new doc makes me very confused. I have no idea how to use pathom. Maybe I will try to read old doc tomorrow.#2020-03-1621:24eoliphantcouple things, i think you want your mutation to return a single ident for the updated shop [:shop/uuid 1111]#2020-03-1621:26eoliphantthen have your resolver output just be [:shop/uuid :shop/engine] and just return (shop-biz/get-shops uuids)#2020-03-1710:54pithylessHej @kwladyka šŸ‘‹ Long time no see. šŸ˜‰ I've attached some notes on how I think about resolvers:#2020-03-1713:17kwladykahello @pithyless šŸ™‚ Very long time ! Thanks for example. I will relocate to Wrocław soon. Give me some news from your live šŸ™‚ (maybe not on this channel šŸ˜‰ )#2020-03-1720:23myguidingstar@wilkerlucio is it possible to apply the batch transformer to a dynamic resolver?#2020-03-1721:36kwladyka
(pc/defresolver get-shop [env {:shop/keys [uuid]}]
  {::pc/input #{:shop/uuid}
   ::pc/output [:shop/uuid :shop/name :shop/engine :shop/config]}
  (shop-biz/get-shop uuid))
(utils/request-eql [{[:shop/uuid "99999999-9999-9999-9999-999999999999"] [:shop/uuid :shop/name :shop/engine :shop/config]}])
=>
#:shop{:uuid "99999999-9999-9999-9999-999999999999",
       :engine :com.wsscode.pathom.core/not-found,
       :name :com.wsscode.pathom.core/not-found,
       :config :com.wsscode.pathom.core/not-found}
What is the best practice? Can I return clear information when the shop is not found? Or is good like it is for EQL?
#2020-03-1721:46kwladykaI feel :com.wsscode.pathom.core/not-found is a bad idea#2020-03-1721:49pithylessIf you return just {:shop/uuid ".."} , then Pathom will try a different resolver that has :shop/engine . So, if you're sure that there is no :shop/engine for this case, you want to return something. You can e.g. return {:shop/engine nil} , but then you need to be sure that whoever is parsing the result will be ok with handling a nil engine correctly. You can also return {:shop/engine :com.wsscode.pathom.core/not-found} and then use a parser plugin (e.g. elide-special-outputs-plugin) to remove the not-found values before returning the result to the client. https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/plugins.html#_built_in_plugins https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/core.cljc#L721-L722#2020-03-1722:18kwladyka
(pc/defmutation update-shop [env params]
  {::pc/sym 'shop/update
   ::pc/params #{:shop/name :shop/engine :shop/config}
   ::pc/output [:shop/uuid]}
  (shop-biz/update-shop params)
  (-> (select-keys params [:shop/uuid])
      (update :shop/uuid str))
  {:shop/uuid #uuid"11111111-1111-1111-1111-111111111111"})
I found resolvers are not called with #uuid but works without.
(pc/defresolver get-shop [env params]
  {::pc/input #{:shop/uuid}
   ::pc/output [:shop/name :shop/engine :shop/config]
   ::pc/transform pc/transform-batch-resolver}
  (println "foo" (shop-biz/get-shops (set (map :shop/uuid params))))
  (shop-biz/get-shops (set (map :shop/uuid params))))
I don’t see println with #uuid . Why?
#2020-03-1722:24kwladykaso…
:shop/uuid #uuid"11111111-1111-1111-1111-111111111111"
if I use #uuid in EQL query I have to return str. But if I use string in EQL query
:shop/uuid "11111111-1111-1111-1111-111111111111"
I can return #uuid or str This is a little unpredictable
#2020-03-1808:15ouvasamHi, i don't know how to create resolvers to do the following
(pc/defresolver api-1 [{:keys [root-query] :as env} params]
                {::pc/output [:api/uri]}
                (let []
                  {:api-1/uri "uri"}))

(pc/defresolver api-items-1 [{:keys [root-query] :as env} params]
                {::pc/input #{:api-1/uri}
                 ::pc/output [:items [[:key1
                                       :key2]]]}
                (let []
                  {:items [{:key1 1
                            :key2 2}]}))


(pc/defresolver api-2 [{:keys [root-query] :as env} params]
                {::pc/output [:api-2/uri]}
                (let []
                  {:api-1/uri "uri"}))

(pc/defresolver api-items-2 [{:keys [root-query] :as env} params]
                {::pc/input #{:api-2/uri}
                 ::pc/output [:items [[:key3
                                       :key4]]]}
                (let []
                  {:items [{:key3 3
                            :key4 4}]}))

;-> this is what i would like to obtain 
{:api-1/uri "uri"
 :api-2/uri "uri"
 :items [{:key1 1
          :key2 2
          :key2 3
          :key2 4}]}
#2020-03-1808:16ouvasamI am completly lost, trying lot of things but can't find a good way#2020-03-1808:17ouvasamIf someone has a tip for this ? Many Thanks#2020-03-1808:22cjmurphyOne quick thing - input (`::pc/input`) is usually a set.#2020-03-1808:24ouvasamyes sorry i did try to reproduce an exmaple with fake code so it is a set in my code sorry#2020-03-1808:25ouvasamthanks#2020-03-1808:25ouvasamMy query looks like this
[:api-1/uri
 {:item
  [[:key1
    :key2
    :key3
    :key4]]}]
#2020-03-1808:30cjmurphyDid you put debug in each of them to see which ones are 'being fired'?#2020-03-1808:34cjmurphyBut just looking at api-1 your output does not match what is being returned. Same with api-2.#2020-03-1808:35ouvasamYes once :items is returned the other is not fired#2020-03-1808:36ouvasamand api-2 should return api-2/uri made a typo in the fake code#2020-03-1809:06ouvasamDon't know if it is a good way but this now work :
(pc/defresolver api-1 [{:keys [root-query] :as env} params]
                {::pc/output [:api/uri]}
                (let []
                  {:api-1/uri "uri"}))
(pc/defresolver api-items-1 [{:keys [root-query] :as env} params]
                {::pc/input #{:api-1/uri}
                 ::pc/output [:items [[:key1
                                       :key2]]]}
                (let []
                  {:items [{:key1 1
                            :key2 2}]}))
(pc/defresolver api-2 [{:keys [root-query] :as env} params]
                {::pc/output [:api-2/uri]}
                (let []
                  {:api-1/uri "uri"}))
(pc/defresolver api-items-2 [{:keys [root-query] :as env} params]
                {::pc/input #{:api-2/uri}
                 ::pc/output [:key3 :key4]}
                (let []
                  {:key3 3
                   :key4 4}))
#2020-03-1809:06ouvasamIf there is a better way to do that i'm interested Thanks,#2020-03-1809:24ouvasamthis is not the good way ...#2020-03-1809:54pithylessThat looks about right, if you want to achieve:
{:api-1/uri "uri"
 :api-2/uri "uri"
 :items [{:key1 1
          :key2 2
          :key2 3
          :key2 4}]}
You can probably hide some of the boilerplate (if you wish) by dropping down to pathom.connect/resolver and generating the resolvers dynamically, instead of using the macros.
#2020-03-1809:55pithyless(although in your psuedocode, api-items-2 needs to also have the nested :items output)#2020-03-1809:58ouvasamMany thanks @pithyless if i add items in the second one it is not called#2020-03-1810:00pithylesswith an eql query: [:api-1/uri :api-2/uri {:items [:key1 :key2 :key3 :key4]}] ?#2020-03-1811:16ouvasamYes it was the case With you’re recommandations I have a solution I think I’ll post it once sure it works #2020-03-1811:16ouvasamThanks #2020-03-1812:11ouvasami have something like that work i just have to add test on api-1 . api-2 to see if some corresponding keys are requested or not since tey are systematically called. But is ok#2020-03-1812:11ouvasam
(pc/defresolver api-sales [{:keys [root-query] :as env} {:keys [api]}]
                {::pc/output [:api-1/id
                              :api-2/id]}
                (let [api-id :
                      [uri label ns-path] (first (acrux/api-details api-id))]
                  (println "sales-resolver")
                  {:api-1/id 1
                   :api-2/id 2}))
; (println "ennv" @(:com.wsscode.pathom.core/entity env))


(pc/defresolver api-1 [{:keys [group-by] :as env} params]
                {::pc/input #{:api-1/id}
                 ::pc/output #{:items1 [:key1 :key2 :store]}}
                (let []
                  (println "api1" params)
                  {:items1 [{:store :b :key1 3 :key2 4} {:store :a  :key1 3 :key2 4}]}))

(pc/defresolver api-2 [{:keys [group-by] :as env} params]
                {::pc/input #{:api-2/id}
                 ::pc/output #{:items2 [:key2 :key4 :store]}}
                (let []
                  (println "api2" params)
                  {:items2 [{:store :a :key3 3 :key4 4} {:store :b :key3 3 :key4 4}]}))

(defn group-by-key
  [items]
  (group-by :store items))

(pc/defresolver api-all [{:keys [group-by] :as env} params]
                {::pc/input #{:items1
                              :items2}
                 ::pc/output #{:items}}
                (let [items1 (group-by-key (:items1 params))
                      items2 (group-by-key (:items2 params))
                      result (map (fn [[k  v]]
                                    (apply merge v))
                                  (merge-with into  items1 items2))]
                  (println "api-all")
                  {:items result}))
#2020-03-1812:11ouvasamthe strange thing is that i can't do a group-by inside the resolver api-all. I should do it in another function ???#2020-03-1812:12ouvasamI use 2.3.0-alpha5#2020-03-1812:41pithylessI don't see why you need api-all at all. I think the problem is that Pathom is not recognizing the output join:
::pc/output #{:items1 [:key1 :key2 :store]}
should rather be:
::pc/output [{:items [:key1 :key2 :store]}]
#2020-03-1812:44pithylessAlso, is :api-1/id and :api-2/id something you want to return to the client? Or is it just a temporary variable you're passing around to resolve the other keys? IIUC, you have two apis that use different identifiers for the same "thing", but return a different subset of information about the "thing".#2020-03-1812:56ouvasamthese two keys should also return the url api to call#2020-03-1812:59ouvasamWhen i do what you suggest, api-2 is not called#2020-03-1813:05ouvasamyes that's it i have multiples api that return differents informations and i need to consolidate these data together. (e.g. on store key)#2020-03-1813:08ouvasamwith this code
(pc/defresolver api-1 [{:keys [root-query] :as env} {:keys [api]}]
                {::pc/output [:api-1/id
                              :api-1/uri]}
                (let []
                  (println "api-1")
                  {:api-1/id 1
                   :api-2/uri "uri1"}))
(pc/defresolver api-2 [{:keys [root-query] :as env} {:keys [api]}]
                {::pc/output [:api-2/id
                              :api-2/uri]}
                (let []
                  (println "api-2")
                  {:api-2/id 1
                   :api-2/uri "uri2"}))

(pc/defresolver api-1-items [{:keys [group-by] :as env} params]
                {::pc/input #{:api-1/id}
                 ::pc/output [{:items [:key1 :key2 :store]}]}
                (let []
                  (println "api-1-items" params)
                  {:items [{:store :b :key1 1 :key2 2}
                           {:store :a  :key1 1 :key2 2}]}))

(pc/defresolver api-2-items [{:keys [group-by] :as env} params]
                {::pc/input #{:api-2/id}
                 ::pc/output [{:items [:key3 :key4 :store]}]}
                (let []
                  (println "api-2-items" params)
                  {:items [{:store :a :key3 3 :key4 4}
                           {:store :b :key3 3 :key4 4}]}))
only api-sales and api-2 resolvers are called. so if i have a query like this
[{:items [:key1 :store :key3]}]
the result is:
{:items [{:key1 1, :store : :key3 :com.wsscode.pathom.core/not-found} {:key1 1, :store :a, :key3 :com.wsscode.pathom.core/not-found}]} 
#2020-03-1813:23ouvasamhere is the parser
(def parser
  (p/parallel-parser
   {::p/env     {::p/reader [p/map-reader
                             pc/parallel-reader
                             pc/ident-reader
                             p/env-placeholder-reader]
                 ::p/placeholder-prefixes #{">"}}
    ::p/mutate  pc/mutate-async
    ::p/plugins [(pc/connect-plugin {::pc/register app-registry})
                 p/error-handler-plugin
                 p/trace-plugin]}))
#2020-03-1814:18pithylessOK, I've tried to come up with a minimal case:
(pc/defresolver api-1 [env input]
  {::pc/output [:api-1/id]}
  {:api-1/id 1})

(pc/defresolver api-1-items1 [env input]
  {::pc/input #{:api-1/id}
   ::pc/output [{:items [:key1]}]}
  {:items [{:key1 1}]})

(pc/defresolver api-1-items2 [env input]
  {::pc/input  #{:api-1/id}
   ::pc/output [{:items [:key2]}]}
  {:items [{:key2 2}]})
This does not work as I would have expected:
[:api-1/id {:items [:key1 :key2]}]
=> ignores :key1 if all 3 resolvers are in parser; will return :key1 if api-1-items2 is not in parser
Never hit this "bug", because I never modeled split-joins this way. Maybe @wilkerlucio can weigh in?
#2020-03-1814:18pithyless(Tested with pathom 2.2.30)#2020-03-1814:27souenzzo@pithyless in my mind model, it's expected I think that app-items should return [{:items [:id]}] then you do :id -> :key1 and :id -> key2#2020-03-1814:29pithylessThat would work and probably how I would have done it myself; but still surprised it does not follow the depedency-graph, but instead probably stops on first resolver that returns something with :items. I would have expected that joins would not short-circuit like that. ĀÆ\(惄)/ĀÆ#2020-03-1814:38pithylessSo, for clarity; the suggestion @ouvasam is to perhaps model it like this:
(pc/defresolver api-items [env input]
  {::pc/output [{:items [:api-1/id :api-2/id]}]}
  {:items [{:api-1/id 1
            :api-2/id 2}]})

(pc/defresolver api-1 [env input]
  {::pc/input #{:api-1/id}
   ::pc/output [:key1 :key2]}
  {:key1 1
   :key2 2})

(pc/defresolver api-2 [env input]
  {::pc/input  #{:api-2/id}
   ::pc/output [:key3 :key4]}
  {:key3 3
   :key4 4})
And then you query like this:
[{:items [:key1 :key2 :key3 :key4]}]

  [{:items [:api-1/id :api-2/id :key1 :key2 :key3 :key4]}]
#2020-03-1814:39ouvasamI did something like this also but once :items is return it does not go throught the other resolvers#2020-03-1814:40pithyless^ I tested the above code in my repl.#2020-03-1814:40ouvasamah ok sorry i'll give it a try now#2020-03-1814:41pithyless
[{:items [:api-1/id :api-2/id :key1 :key2 :key3 :key4]}]
=> {:items [{:api-1/id 1, :api-2/id 2, :key2 2, :key1 1, :key3 3, :key4 4}]}
#2020-03-1814:47ouvasamMy problem is that api-1 api-2 return both a list of [`{:key1 1 :key2 2}]` not just a single hash-map#2020-03-1814:48ouvasamthe more i think about this, the more the solution with a global resolvers that receive items1 and items2 to merge the results should be the safier#2020-03-1814:50ouvasammany thanks for taking some time for this !#2020-03-1814:57pithylessDo you mean modifying it for batch requests?
(pc/defresolver api-1 [env input]
  {::pc/input #{:api-1/id}
   ::pc/output [:key1 :key2]
   ::pc/transform pc/transform-batch-resolver}
  (mapv #(fn [{:keys [:api-1/id]}]
           {:key1 1
            :key2 2})
        input))
Or that for a single api-1/id there will be multiple key1 's? That I would just model by returning it as a map of key1 => [values]
#2020-03-1815:05ouvasamin my previous example, i d othe following api-1 return the uri from a db to get the items. api-1-items, use this uri to call an api and get the a list of keys (key1, key2). With this i do a single call for api1. A s there is lot of apis that return differents keys. if some keys from a different api are requested i'd like to do only one call by api. So api-2 work the same as api-1 but with a different uri and different keys. As i watched your video about pathom. I retain "FLAT" which is exactly what i need for the rest of the app. using namespaces keys is really interesting. Hope i am clear ? Another problem i have is that, all these apis don't return the same length of list. So i should be able to consolidate data in specific manners. That's why i think a global resolver should do the trick#2020-03-1815:12pithylessFlat namespaced keys for different apis is IMO a good way to approach this problem.#2020-03-1815:16ouvasamThat is what i should do to prove that to my customer. I can make it work with this global resolver but i am new to pathom so if there is better solution to do i am interested#2020-03-1815:17ouvasamI did use triple stores for other jobs and uri's were very powerful#2020-03-1815:18ouvasamAnd thanks for your videos !#2020-03-1817:17adamfeldmanPlease share links if you can šŸ™‚ I only found ā€œDomain Modeling with Datalogā€ https://www.youtube.com/watch?v=oo-7mN9WXTw&amp;t=9s#2020-03-1906:36ouvasamhttps://www.youtube.com/watch?v=UvJEBMOtayk#2020-03-1815:21pithylessThanks, always nice to hear someone is getting something out of them. 😊 Need to go afk for a bit, good luck with the modeling ;]#2020-03-1815:22ouvasamThanks !#2020-03-1922:22kwladykaHow do you return errors for request? let say somebody send uuid with value abc which is invalid data. I want to return something like
{:logs [{:type :error,
         :code :validation,
         :message {[:shop/uuids :shop/uuid] "UUID has to be in valid format (for example 00000000-0000-0000-0000-000000000000).",
                   [:shop/uuids] "foo"}}]}
Do I have to add this to each ::pc/output ?
#2020-03-1922:35souenzzoYou may implement a custom error-handler-plugin#2020-03-2000:09wilkerlucio@kwladyka you can set ::p/process-error in the env, this allows you to process the error in any way you want to return to the user, example: (p/parser {::p/env {::p/process-error (fn [env error] error)}}) ; returns the literal error#2020-03-2000:09wilkerlucioby default Pathom uses a string representation due to the initial usages always crossed the wire, so that ended up becaming the default#2020-03-2000:10wilkerluciobut as experience shows, its rare that this default is the best option#2020-03-2013:36kwladyka
(def parser
  (p/parallel-parser
    {::p/env {::p/reader [p/map-reader
                          pc/parallel-reader
                          pc/open-ident-reader
                          p/env-placeholder-reader]
              ::p/process-error (fn [env error] (println "foo" error))
              ::p/placeholder-prefixes #{">"}}
     ::p/mutate pc/mutate-async
     ::p/plugins [(pc/connect-plugin {::pc/register resolvers})
                  p/elide-special-outputs-plugin
                  p/error-handler-plugin
                  p/trace-plugin]}))
it doesn’t work for me. ::p/process-error never triggered
#2020-03-2013:42cjmurphyThis plugin (put under ::p/plugins) works for me.#2020-03-2013:43souenzzop/error-handler-plugin should be before p/elide-special-outputs-plugin#2020-03-2013:44kwladyka@souenzzo still doesn’t work when I return map . ::p/process-error work when I return string#2020-03-2013:47wilkerluciocheck if it isn't some encoding issue, but if you run on the REPL it should give the correct answer#2020-03-2013:47kwladykait is REPL#2020-03-2017:35wilkerluciostrange, what is getting out on the error when you try to use a map?#2020-03-2018:32kwladykathere is no error, but also no data, so {} is returned#2020-03-2018:33kwladykaI have to add :logs to ::pc/output to see this data#2020-03-2018:33kwladykabut if not, then there is no error and no data#2020-03-2018:33kwladyka{:logs :com.wsscode.pathom.core/not-found, :shops :com.wsscode.pathom.core/not-found}#2020-03-2019:05kwladyka@wilkerlucio any idea how can I return this :logs without adding to ::pc/output in each resolver?#2020-03-2019:06kwladykaI don’t want to throw exception for things which I predicted. This is well precited error in code, so no reason to throw exception.#2020-03-2019:08kwladykathis is the thing which probably we misunderstand (maybe)#2020-03-2019:09kwladykaat least pathom works like that#2020-03-2019:10kwladykaI want to return :logs even if not in ::pc/output and even if not listed in EQL query#2020-03-2019:14wilkerlucio@kwladyka seems like you are talking about something different than error processing#2020-03-2019:14wilkerlucioif you don't likst things on ::pc/output, the resolver is not going to get called#2020-03-2019:14kwladykasure and this is fine#2020-03-2019:14wilkerlucioand I dont see any way around that, this is a very basic premise of pathom, because this is how things are indexed and how pathom decides what to trigger#2020-03-2019:14kwladykaI will show you:#2020-03-2019:14kwladykathis is ver. which I have now:#2020-03-2019:14wilkerluciobut maybe I'm not understading you yet#2020-03-2019:15kwladyka
(pc/defresolver get-shops [env {:shop/keys [uuids]}]
  {::pc/input #{:shop/uuids}
   ::pc/output [{:shops [:shop/uuid :shop/name :shop/engine :shop/config]}]}
  (let [shops (shop-db/get-shops-by-uuid (case uuids
                                           :all :all
                                           (set (map (partial safe-coercion uuid/as-uuid) uuids))))]
    (if (vector? shops)
      {:shops (mapv #(update % :shop/uuid str) shops)}
      shops)))
#2020-03-2019:15kwladykaBUT if during processing all of this there is for example error in validation then this resolver return:#2020-03-2019:15kwladyka
{:logs [{:type :error,
         :code :validation,
         :cause ({:via [:shop/uuids :shop/uuid],
                  :val "999",
                  :message "UUID has to be in valid format (for example 00000000-0000-0000-0000-000000000000)."}
                 {:via [:shop/uuids],
                  :val #{"999"},
                  :message "Has to be set of uuid (for example #{00000000-0000-0000-0000-000000000000})."})}]}
#2020-03-2019:15kwladykainstead of :shops#2020-03-2019:16kwladykaso the resolver doesn’t have to be called for :logs request#2020-03-2019:16kwladykaI just want to have 1 unified clear way of returning errors#2020-03-2019:16kwladykabut not exceptions, errors which my code predict and can react#2020-03-2019:16wilkerluciothat's quite a different approach, nothing wrong with that, but requires you to re-invent the error system#2020-03-2019:16kwladykaexceptions are things which goes wrong#2020-03-2019:16wilkerluciopathom errors are just data in the end#2020-03-2019:17wilkerlucioif you want something like that, you have to make your own plugin#2020-03-2019:17wilkerlucioadd an atom to the env to accumulate things#2020-03-2019:17kwladykaDo you see better way to achieve something similar?#2020-03-2019:17wilkerlucioand in the ::p/wrap-parser plugin you can inject the collected things to the final response#2020-03-2019:18wilkerluciohow are you using this? what your query looks like?#2020-03-2019:19kwladykafor example this is wrong query because of the validation
(utils/request-eql [{[:shop/uuids #{"999"}] [:shops]}])
#2020-03-2019:19kwladykathis is good one
(utils/request-eql [{[:shop/uuids #{"00000000-0000-0000-0000-000000000000"}] [:shops]}])
#2020-03-2019:20kwladykaand I want to return in some way errors during processing of this wrong query#2020-03-2019:20kwladykaso:#2020-03-2019:20kwladyka{:logs [{:type :error, :code :validation, :cause ({:via [:shop/uuids :shop/uuid], :val ā€œ999ā€, :message ā€œUUID has to be in valid format (for example 00000000-0000-0000-0000-000000000000).ā€œ} {:via [:shop/uuids], :val #{ā€œ999ā€}, :message ā€œHas to be set of uuid (for example #{00000000-0000-0000-0000-000000000000}).ā€œ})}]}#2020-03-2019:20wilkerlucioI personally don't use collections as inputs, I found that given then a name is usually a better option (if possible)#2020-03-2019:20wilkerluciojust to understand your case better, how do you build this set to make this query? (whats the source for this list?)#2020-03-2019:21kwladyka> I found that given then a name is usually a better option (if possible) What do you mean?#2020-03-2019:21kwladykathis query postgresql#2020-03-2019:21wilkerlucioI mean, before, when you build the query. I imagine there is some place from which you get this set of ids#2020-03-2019:22kwladykaI can get all shops by
(utils/request-eql [{[:shop/uuids :all] [:shops]}])
#2020-03-2019:22kwladykaMaybe I will change this later to separate resolver, will see#2020-03-2019:23wilkerlucioso, what I would do there is [{:shop/all [:shop/name]}]#2020-03-2019:23kwladykabut still the issue is I don’t know how can I return good feedback why query failed#2020-03-2019:23wilkerluciobecause, lists tend be something else#2020-03-2019:23wilkerlucioyou are using it in a peculiar way#2020-03-2019:23wilkerlucioidents should refer to a single entity, that's more a Fulcro premise, but a lot of Pathom is built upon this idea#2020-03-2019:24kwladykawhat if I will have :shop/all as real value in :shop map data? It will make a little confusion.#2020-03-2019:24wilkerlucioso, instead of trying to send them by hand on an ident, you can have a resolver that returns all the ids (like: [{:shop/id 1} {:shop/id 2} {:shop/id 3} ...])#2020-03-2019:25kwladykaThis is why I was thinking about not use /all to avoid confusion it is a value of :shop#2020-03-2019:25wilkerlucio@kwladyka this is a very open graph, you should be consistent about what properties mean, so having it as both would be wrong modeling#2020-03-2019:25wilkerluciobut you can give other names, as long as they are consistent#2020-03-2019:25kwladykahttps://clojurians.slack.com/archives/C87NB2CFN/p1584732290329700?thread_ts=1584711881.316000&amp;cid=C87NB2CFN I can do this too#2020-03-2019:26wilkerluciosorry, I don't have a lot of time to chat around this now, gotta get back to work, but I suggest you try to follow the things as documented, at least until you get a better feeling of pathom, the direction you are taking is just unknown and likely not well supported#2020-03-2019:26kwladykaI am experimenting a little#2020-03-2019:26kwladykabut so far I stuck on :logs so the feedback why query failed (for example data validation)#2020-03-2019:28wilkerlucioyou are kind of manually dealing with list processing in this approach, pathom has a bunch of helpers to deal with sequences (think it like jQuery), also, errors in pathom are made in a way to be tolerant as much as possible#2020-03-2019:29wilkerlucioI feel like you are thinking like entities (which is very common, that's what happens almost everywhere else) but in Pathom it really helps if you try to get your head around properties (not entities) as the primary building block#2020-03-2019:29wilkerlucioso, when you think about something failing, its not the entity, but each attribute#2020-03-2019:29wilkerlucioeach attribute may fail for a different reason, so you can get partial results#2020-03-2019:30wilkerlucioso, in my view, its not about the :shop/id failing, its about the detail of it failing because the dependency :shop/id was invalid, so the error would be on the final attribute (`:shop/name` or something)`, not on the list processing#2020-03-2019:30wilkerlucioI really gotta go now, but would be happy to chat more around this some other time#2020-03-2019:34kwladykaI see. But it makes it hard for API client processing. The client has to understand when the value is an error and when expected value.#2020-03-2019:34kwladykaUnless I miss something#2020-03-2019:34kwladykaBut what you are saying make sense#2020-03-2019:35kwladykaOnly I don’t see how to make this easy to parse byĀ client and understand when the client can use reposne and when not#2020-03-2022:17kwladykaso after all… are you saying throwing exceptions is the right way of returning validations errors in pathom?#2020-03-2022:17kwladykaI can’t find good way of doing this in pathom#2020-03-2022:44kwladykahmm with exceptions way it can be much simpler in pathom#2020-03-2018:11myguidingstar@wilkerlucio is it possible to apply the batch transformer to a dynamic resolver?#2020-03-2018:11wilkerlucioI think it should be, but I didn't get to play with that on dynamic resolvers yet#2020-03-2018:13myguidingstaryou mean it's already supported? how?#2020-03-2018:13wilkerlucioI just did a check, its not supported yet#2020-03-2018:13wilkerluciobut its on my list, if you have a close case you need it, I can prioritize it#2020-03-2018:14wilkerlucioits not a hard thing to add#2020-03-2018:15myguidingstarI'm converting walkable to a dynamic resolver, it'll be much easier if it's supported#2020-03-2018:16wilkerluciook, tomorrow I'll have some time, I'll try to add it, I think the basic support (like the previous ones does) should be strait forward to implement#2020-03-2018:16wilkerlucioI also would like to support multi-depth batching in this new one, that may take more work#2020-03-2018:17myguidingstareven when you can't implement it soon, please give me several lines of example code so I know what to expect when it's done#2020-03-2018:18wilkerluciohard to tell with certain without sitting and giving some thinking, but I think it will be pretty much like the current one, instead of getting a map as input, you will get a sequence of maps (the inputs to batch)#2020-03-2018:19myguidingstarwhat is "multi-depth batching"? Say [{:a [{:b [:c]}]}], does it mean all the :c children from multiple :b branches?#2020-03-2018:20wilkerlucioyeah, like: [{:people [{:person/family [:person/name]}]}]#2020-03-2018:20wilkerlucioI had people asking about this before, not in the near future, but something to think about how to implement over time#2020-03-2119:42wilkerluciostarted looking into it, I found that batch already works for regular resolvers on reader3, the issue is that batch relies on caching, and dynamic resolvers don't support caching due to their complex input/output relations#2020-03-2119:42wilkerlucioI'm starting to think about how we can go around that for dynamic resolvers#2020-03-2120:15myguidingstarWhat kind of caching? I use p/cached in my dynamic resolver and it seems to work#2020-03-2120:18wilkerluciop/cached works, but batch relies on the standard resolver caching, which doesn't work for dynamic, the reason is, when we do a batch for regular resolvers, we get the batched result, and then cache each entry as if they were called separated, so when the processing continues and the user reach that same resolver, the batch will have the call cached, this way it goes fast#2020-03-2120:19wilkerluciobut for dynamic resolvers that's much harder, given there is not well defined input to cache from a dynamic response (what is the input? what if the request was already composed from multiple things on the dynamic, how to isolate that part?)#2020-03-2120:19wilkerluciomakes sense?#2020-03-2120:24myguidingstarYes#2020-03-2120:27myguidingstarBtw I've made progress on converting walkable to a dynamic resolver. The batching was done by leveraging p/cached that makes batched query from parent node :)#2020-03-2120:59wilkerlucionice!#2020-03-2019:29kwladykaHow do you return errors (not exception ,but for example spec validations info) from resolvers?#2020-03-2019:29yendaFor a search resolver, should a partial name be rather an input or a parameter?#2020-03-2019:31wilkerlucioI usually do as a param#2020-03-2019:37yendayeah it was my first intuition but the issue is that the search is not part of the response#2020-03-2019:38yendawould that make sense to return it as a field then?#2020-03-2019:42wilkerlucioif you need in on the response as well, yes, you can do that#2020-03-2117:16kwladykaDo you have ready solution to share for :com.wsscode.pathom.core/errors as EDN data instead of string?#2020-03-2117:42kszabo::p/process-error (fn [env error] (Throwable->map error))#2020-03-2119:48wilkerlucioHello folks, I would like to give more visibility around the work I do on Pathom and also on the rest of my open-source work. To improve this situation I've been working on a knowledge base for all my work, the idea is to use this so I can quickly add information and link related things, so you will see data around Pathom, around EQL, all linked and easy for me to update (meaning that will encourage me to update more often). On top of a knowledge base, this also works as journaling, so if you want to see the most recent stuff that I'm working on or thinking of, that's also a great place to check that! And finally, here is the link for this knowledge base: https://roamresearch.com/#/app/wsscode/page/WWmdSMHKY I would love to hear what you think about this format, any suggestions are welcome. Cheers!#2020-03-2121:21kszaboseems like a very useful tool, thanks for introducing it#2020-03-2604:56Abhinav SharmaPinned it to the channel#2020-03-2120:08myguidingstarThansks for sharing Wilker#2020-03-2121:01kwladyka
::p/process-error (fn [env error]
                                  (let [data (ex-data error)]
                                    (if (= :validation (:code data))
                                      (do (l/debug error)
                                          (update data :cause (partial keep spec-problems->errors)))
                                      (do (l/error error)
                                          {:type :error
                                           :code :unknown
                                           :cause "Unknown exception. The exception is logged and will be fixed."}))))
So far I have this if somebody is interested in. Finally I got conclusion I have to use exceptions to return any validation issue in pathom and it works well so far.
#2020-03-2121:41kwladyka
(pc/defresolver all-users [{::keys [db]} _]
  {::pc/output [{:user/all [:user/id :user/name :user/email :user/created-at]}]}
  (vals (get @db :users)))
why :user/all vs :users? :user/all confuse of value all in user data. Is it some kind of standard in pathom? Can I use :users?
#2020-03-2121:49kszaboYou can use :users if you want.#2020-03-2121:50kszaboIt’s a domain modelling problem which is not a black-and-white situation#2020-03-2209:27kwladykaCan I change :com.wsscode.pathom.core/not-found to another key in answers in pathom?#2020-03-2211:13pithylessJust write your own pathom plugin that walks the final output, similar to elide-not-found: https://github.com/wilkerlucio/pathom/blob/cb8cb1c35e19d4956778157ee280b5ebe90cab4e/src/com/wsscode/pathom/core.cljc#L239-L249#2020-03-2212:55kwladykaYes, I can do this even with final output without plugin. I have an issue to understand settings and possibilities for configuration.#2020-03-2213:27kwladykaoh there are readyĀ plugins, I have to read pahtom code one day#2020-03-2212:55kwladykaDo you use fulcro ? Why? Why not?#2020-03-2212:56kwladykaI am trying to make a decision if I want to to use fulcro or not. I feel it is too much framework, but if I want to use pathom maybe it makes sense anyway?#2020-03-2213:08cjmurphyIt is Fulcro on the client then often Mount, Ring, Pathom on the server. It is not Fulcro or Ring, which is I think what you were saying. And once you have a server that takes EQL, why not use a client that sends EQL?#2020-03-2213:09kwladykamy mistake then, but the point is if use fulcro or not#2020-03-2213:09kwladykaSure the client which send EQL make sense#2020-03-2213:10kwladykabut still not sure about fulcro . I didn’t use this one, but it looks like framework which make many decisions for me which I can’t easy change.#2020-03-2213:10kwladykaSo do use use fulcro? Why?#2020-03-2213:11kwladykaI guess I can use EQL queries without fulcro#2020-03-2213:13kwladykafulcro is BE and FE right? So have I to use this everywhere?#2020-03-2213:13cjmurphyFulcro composes the EQL that you provide to the components you create. You can still use what you want for CSS for example. You can still use JavaScript libraries. Fulcro is good for big applications as your code is declarative.#2020-03-2213:14cjmurphyNo Fulcro is front end only. It used to be back-end as well, until Pathom came along.#2020-03-2213:14kwladykaBut for example I have to use shadow-cljs ? At least doing this in different way will make everyhing harder#2020-03-2213:14kwladykaoh I didn’t know#2020-03-2213:14kwladykaso fulcro is FE only now#2020-03-2213:14kwladykaok it changes a lot#2020-03-2213:14cjmurphyYes.#2020-03-2213:15kwladykagreat, thanks for clarification!#2020-03-2213:16cjmurphyNow the pessimistic part of a Fulcro mutation is handled by Pathom resolver.#2020-03-2213:16kwladykagood, then maybe I will take a look on Fulcro once again#2020-03-2213:18cjmurphyThere's many layers to Fulcro. Start with app state. That may be all you need.#2020-03-2213:22kwladykaso far I prefered re-frame#2020-03-2213:24kwladykaI don’t know too much about fulcro#2020-03-2213:26cjmurphyThere is a book and you have a big head-start if you are already comfortable with EQL: http://book.fulcrologic.com/#_queries_eql#2020-03-2213:39kwladykayes…. very long doc to start with fulcro#2020-03-2213:45cjmurphyIts a reference guide. But the app state part (normalisation) is really very simple and should give you the equivalent of a re-frame application, but without writing 'subscription' style code.#2020-03-2213:47cjmurphy(I say it is simple, but it does take some effort to pick it up).#2020-03-2220:00mdhaneyI would disagree that Fulcro is more ā€œframeworkeyā€ than re-frame. Fulcro is primarily concerned with 1) managing app state transitions and 2) providing components with access to relevant state. This is really not that different from re-frame. Both share a lot of the same concepts around #1, but use different mechanisms for #2 (queries vs. subscriptions). Fulcro arguably provides more out of the box for #1, especially when it comes to remote interaction, but it doesn’t tie your hands - pretty much everything in the transaction processing pipeline is pluggable. And the things that tend to change the most are the easiest to modify. So for example, while in theory you could replace the whole transaction pipeline with something else, that would be a lot of work (and fortunately, rarely needed). However, replacing the default merge logic (which is probably sufficient for 90% of cases) with a specialized merge for a single transaction that requires special consideration is relatively easy. Or customizing error handling for a single transaction - piece of cake. One of the 4 built-in renderers doesn’t fit your needs - write your own that does? This may sound daunting, but take a look at the source code for one of the existing renderers - there’s not much there! The whole architecture of Fulcro has been refined and simplified over several iterations (with F3 finally getting rid of the old remaining Om-Next baggage). Whenever you need to customize something - which isn’t often, but will happen if you write enough ā€œrealā€ apps - the clean architecture and using the existing implementation as a reference makes this much less painful than you would think.#2020-03-2220:11kwladykaDisclaimer: what I said was about my feelings 1 year ago (I guess). So it can be not up to date.#2020-03-2311:39tvaughanWe just went through an evaluation of the current "state-of-the-art" to see if we should continue with our current prototype or pivot. This is what we came up with (somewhat abbreviated): #2020-03-2311:39tvaughan
As a business building an extremely ambitious product on a tight budget, we
want to be able to:

* Focus as much time and thought on building features that bring value to our
  customers, rather than, for example, building a bespoke protocol between the
  browser and back-end if a better alternative is available.

* Share early iterations of our application with everyone in our organization
  easily. Capture feedback and push out new iterations quickly.

* Be able to support new customers immediately. Our feature road-map will
  always be in constant flux, but at no point should our ability to make
  changes with confidence be compromised. In addition to feature development,
  tests and automation must be priorities. Our development and production
  environments must match.

* Be ready to onboard new team members. "Works on my laptop" is never
  acceptable. Enough of our application must be documented and automated so
  that anyone with a technical background "reasonably proficient" in our work
  can contribute without requiring hand-holding. Or course, we'd never turn
  down a request for help.

Does our current implementation help or frustrate our attempt to meet these
objectives? Assuming the answer is "frustrate," what should our next steps be?

Our new challenge now is to decide what we want to replace in our current
implementation and with what. Our requirements are:

* Clojure, Clojurescript, and Datomic are immutable choices. We love these
  solutions for their simplicity and expressiveness. We believe these are the
  best choices available for achieving our long-term business goals.

* Be measurable. Our work should be guided by data not guesswork.

* Be testable. The entire application should have 100% test coverage as
  measured by its acceptance or integration tests (not unit tests). Changes
  should be made with confidence.

* Be performant. As builders of both the front-end and back-end components, we
  should encourage cooperation between them, not purposefully isolate
  them. Our customers and the end-user experience should be our priorities,
  not artificial divisions between software components or technical teams.
  Each component of the application should cooperate to support progressive
  enhancement where every URL is a real URL that, when requested, returns a
  server-side rendered page view. Further actions by the user result in
  browser-side changes through API calls and minimal component updates.

* Be pragmatic. We should be eager to innovate when the need arises. However,
  we should be humble enough to understand that sometimes, as software
  developers, we can become enamored with new toys that are entertaining, and
  possibly educational, but offer little business value. We should choose
  existing solutions to our problems that are acceptable enough over building
  our own solutions simply because we want to. For example, a standards-based
  protocol, like , that allows
  clients to craft their own queries with little to no server-side updates is
  preferable to our current bespoke solution.

* Be documented and supported. Our choices should have active, helpful, and
  inclusive development communities built around them.

Therefore we seek to evolve the current implementation so that it helps,
rather than frustrates, our ability to adhere to these principles. Complex and
incorrect business logic will be simplified and tested. Custom built
components will be replaced with better alternatives.

Our alternatives are:

* Rum, 

  Rum is a solid piece of software, created and maintained by @tonsky, that
  provides a minimal wrapper around react for creating client-side components,
  and supports server-side rendering. Although rum does not provide a protocol
  between client and server, nor a client component state-management solution.

* Reagent, 

  Together with Re-frame, , this is by far the
  most popular choice for building reactive client components in
  Clojurescript. These do not include the server-side solutions we need.

* Hoplon, 

  Hoplon was created in-house at AdZerk, and later spun-out as an open source
  project. Hoplon is a complete solution although it uses a less intuitive
  spreadsheet-like method for building applications with little obvious
  advantage, and would not be compatible with Datomic without considerable
  effort.

* Luminus, 

  Luminus is a complete solution like Ruby-on-Rails, and is tightly coupled to
  SQL. Like Hoplon, Luminus would not be compatible with Datomic without
  considerable effort.

* Pedestal, 

  Pedestal is supported by Cognitect, the custodians of Clojure and the
  creators of Datomic. Except Pedestal is best suited for building server-side
  APIs, and applications that need to stream data in "near real-time" to large
  numbers of concurrently connected clients.

* Fulcro, 

  Fulcro is based on Om.Next, provides both front-end and back-end components
  that work well together, includes tools for building and debugging both,
  supports server-side rendering, and is supported by a diverse
  community. Paid commercial support is provided by the core developer team at
  Fulcro Logic.

* Datomic Cloud, 

  Datomic Cloud is a special version of the Datomic database that is tightly
  integrated with AWS. Datomic Cloud includes a set of Cloud Formation
  templates for running Datomic in AWS, and taking advantage of AWS specific
  features, such as auto-scaling groups and load-balancers for distributing
  traffic across a cluster of Datomic peers. Datomic Cloud also includes Ions,
  , to help run Clojure functions
  as AWS Lambdas. Cognitech claims Datomic Cloud is a suitable framework for
  building complete applications. Unfortunately Datomic Cloud alone is
  insufficient. Datomic Cloud lacks a framework for building and testing
  client-side applications, and a protocol for interacting with Ions.
#2020-03-2311:48kwladykaDid you decide fulcro on the end? Did you think about re-frame?#2020-03-2311:56tvaughanYup, we went with Fulcro (and mount and pathom). I'm pretty happy with it so far. I built a commercial product based on om.next, reagent, and re-frame right around the time om.next was introduced and it looked a lot like Fulcro in the end. I like reagent and re-frame, but I really did want a solution that was "more complete" - without looking like Rails or Django.#2020-03-2316:17eoliphantyeah, my experience as well. picked up fulcro, liked it but felt like it was ā€˜too much’ put it back down, went with re-frame, theen realized that all the helpers I’d written for db organization, api access, etc were a hacky fulcro-lite, picked fulcro back up lol#2020-03-2213:16kwladykaok I got an asnwer: fulcro is FE only now. I remember it as BE and FE solution. So I am fine with this.#2020-03-2300:12myguidingstarWhat is the setup for async dynamic resolver? I tried to make an async version of a working dynamic resolver, but the resolver seems to be executed only once and I don't know why#2020-03-2313:56myguidingstaranother question: I have this ident query [{[:a/id 1] [:a/x :a/y]}] against a dynamic resolver. But the first call to the function has env's :dispatch-key being :a/x while I expect it to be [:a/id 1] (meaning: one level up). Is that intended behavior? What's the recommended way to detect such case because I usually infer from :dispatch-key?#2020-03-2316:23wilkerluciothere is a new data point in the reader3, which is :com.wsscode.pathom.connect.planner/node (on the env), this contains the planner data about the call, you should look into this instead of :ast#2020-03-2316:10myguidingstar@wilkerlucio is there any util functions to convert an index-io to index-oir? (with a fixed resolver sym, of course)#2020-03-2316:24wilkerlucionot possible, they may not match, and have different considerations, so you have to make each differently#2020-03-2316:24wilkerlucioconsider index-io is used only for auto-complete (which is important), but things should still work without it#2020-03-2317:16myguidingstarI figured out I'd better stick to pc/add and have all indexes generated#2020-03-2316:40myguidingstarmany thanks#2020-03-2622:51Jakub Holý (HolyJak)Hello! How to troubleshoot that my resolver isn't called? See https://clojurians.slack.com/archives/C68M60S4F/p1585258283055800?thread_ts=1585258283.055800&amp;cid=C68M60S4F thanks!#2020-03-2623:10wilkerlucioit seems in your query side the :latest-invoice is missing the namespace, in the ::pc/output it is :organization/latest-invoice#2020-03-2709:01Jakub Holý (HolyJak)OMG, thanks a lot! I would need some kind of "idiot mode" that prints warnings when i ask for a key that Pathom does not know about (and has another "similar" one)#2020-03-2712:56wilkerlucioactually, that can be quite easy to do as an extra reader, try this setup and let me know what you think:#2020-03-2712:56wilkerlucio
(defn validate-connect-key-reader [env]
  (let [k         (-> env :ast :key)
        index-oir (-> env ::pc/indexes ::pc/index-oir)]
    (when (and (not (p/ident? k))
               (not (p/placeholder-key? env k))
               (not (contains? index-oir k)))
      (println (str "WARN: trying to read key " k " which is not available in the index.")))

    ::p/continue))

(def parser
  (p/parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/reader3
                                            validate-connect-key-reader
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate
     ::p/plugins [(pc/connect-plugin {::pc/indexes  indexes
                                      ::pc/register registry})
                  p/error-handler-plugin
                  p/trace-plugin]}))
#2020-03-2712:56wilkerlucio
(defn validate-connect-key-reader [env]
  (let [k         (-> env :ast :key)
        index-oir (-> env ::pc/indexes ::pc/index-oir)]
    (when (and (not (p/ident? k))
               (not (p/placeholder-key? env k))
               (not (contains? index-oir k)))
      (println (str "WARN: trying to read key " k " which is not available in the index.")))

    ::p/continue))

(def parser
  (p/parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/reader3
                                            validate-connect-key-reader
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate
     ::p/plugins [(pc/connect-plugin {::pc/indexes  indexes
                                      ::pc/register registry})
                  p/error-handler-plugin
                  p/trace-plugin]}))
#2020-03-2707:23myguidingstarI guess this is a bug with the new planner or dynamic resolver https://gist.github.com/myguidingstar/53f2b552e2a5dca664b5df0d4780f8c7#2020-03-2712:57wilkerluciojust in case, can you please try on the latest SHA on the query-planner branch? I've been doing some fixes this week.#2020-03-2712:57wilkerluciobut Im not 100% on the current way the new planner handles placeholders, so reports like this are very welcome šŸ™#2020-03-2716:51myguidingstar@U066U8JQJ just tested on latest commit, the issue remains. Also, it's not about placeholders. It's in a normal branch. I tried to remove the placeholders, too.#2020-03-2716:58wilkerlucioone thing I noticed, the resolvers you are adding with pc/add, they are just ending as regular resolvers, so they are not working as dynamic things, for them to be dynamic you need to point to the real dynamic resolver using ::pc/dynamic-sym, like foreign does#2020-03-2716:58wilkerlucioalso, be careful around that, the resolvers around dynamic, they do not work like regular resolvers#2020-03-2716:59wilkerluciothey are aid resolvers to help the dynamic resolver figures itself, I surely have to document this better, but I myself need to play and figure them better too#2020-03-2716:59wilkerluciojust keep in mind, they are different things with different expectations#2020-03-2717:14myguidingstar@U066U8JQJ I only leveraged pc/add to build the indexes, after that I use mergeto override the resolver with :pc/dynamic-resolver? true#2020-03-2717:14myguidingstaras seen here https://gist.github.com/myguidingstar/53f2b552e2a5dca664b5df0d4780f8c7#file-planner-clj-L42#2020-03-2717:14myguidingstaris that enough?#2020-03-2717:15wilkerluciono#2020-03-2717:16wilkerluciowhat I'm pointing is not the dynamic resolver itself, but the "children resolver" of that dynamic resolver#2020-03-2717:16wilkerluciohttps://github.com/wilkerlucio/pathom/blob/query-planner/src/com/wsscode/pathom/connect/foreign.cljc#L85#2020-03-2717:17wilkerlucioso, the resolvers that are intended to help a dynamic reoslver, they need to have a ::pc/dynamic-sym (different from ::pc/dynamic-resolver?) that points to the dynamic resolver#2020-03-2717:17wilkerlucioits easy to imagine this in the foreign context, where all resolvers that are foreign (from a separated parser) should point to the dynamic resolver that calls that parser#2020-03-2717:18wilkerluciothey are used to decide how to call the foreign, but they are never part of the execution graph themselves, and to pathom understand that they need to point to the dynamic resolver they are assisting, makes sense?#2020-03-2717:19myguidingstarlet me see, still trying to process šŸ˜‰#2020-03-2717:36myguidingstarwhy don't I see this dynamic-sym`` thing in pathom-datomic?#2020-03-2717:43wilkerluciobecause the datomic is simple enough it doesnt need one, datomic is just too regular šŸ™‚#2020-03-2717:44wilkerlucioif you see, the datomic only has the dynamic resolver, it doesn't have any "assistent resolvers"#2020-03-2717:44wilkerluciographql will require some#2020-03-2717:48myguidingstarwhat are the input and output of an assistant resolver?#2020-03-2717:50myguidingstarI still don't see why (and where) I should add dynamic-sym to the gist link (it's basically me mimicking pathom-datomic, but too lazy to type the indexes so I use p/add)#2020-03-2718:15wilkerlucioso, seems like you are trying to ray-trace how to make it work, which will be dificult IMO, the pathom-datomic is the simplest dynamic resolver that I can imagine, because datomic is very consistent across relationships (any entity can use any property of the schema, very consistent)#2020-03-2718:15wilkerlucioin your case, its not going to be so much#2020-03-2718:16wilkerluciothe need for assistent resolvers is so pathom can understand how to compute nested queries to send to the dynamic resolver (reducing number of calls), because of the nature of deep queries, this defintion can't be done in a single place, and may require multiple passes to figure what to send, that's were the assistent resolvers come, they serve as instructions of what is available at some specific context for that dynamic resolver#2020-03-2718:17wilkerluciothat said, I didn't made any other implementation on this expect for the foreign-parser thing#2020-03-2718:17wilkerluciothe way to use this for other types of dynamic resolvers is unclear at this point#2020-03-2718:18wilkerlucioI hope to get more docs around that in the next month, but right now I'm focusing on tooling and more on the user side of the story (not the extension side for developers of dynamic resolvers, yet)#2020-03-2718:18wilkerluciosorry its so confusing, its just you are a very early adopter of something that still work in progress šŸ˜›#2020-03-2718:19wilkerlucioI believe this will be more figure out when I sit and stop to do the new GraphQL integration, that's the one closest to what you are trying to do#2020-03-2718:21myguidingstarwell, I guess I did it wrong when I tried to port walkable to planner/dynamic by mimicking datomic. It's working now, but I think the resolver is doing too much right now. I guess some assistant should be extracted instead. I will try that#2020-03-2718:21myguidingstarThank you so much for all these nice things šŸ˜„#2020-03-2904:25yendaAm I doing something wrong or pathom return errors differently for resolvers and mutations? I have the following process-error function:
(defn process-error [env err]
  (if (ex-cause err)
    {:error {:cause (ex-cause err)
             :message (ex-message err)
             :data (ex-data err)}}
    (do (log/error "An error occured"
                   {:root-query (::p/root-query env)
                    :error (p/error-str err)})
        {:error {:cause :unexpected-error
                 :message "An error occured"}})))
When I throw an error in a mutation I get this kind of response:
{login
  {:error
   {:cause :login/invalid-email-or-password,
    :message "Invalid email or password",
    :data {:email-or-username "
When I throw an error in a resolver I get:
{:auth/user :com.wsscode.pathom.core/reader-error,
  :com.wsscode.pathom.core/errors
  {[:auth/user]
   {:error
    {:cause :auth/invalid-token,
     :message "Invalid token",
     :data
     {:token "bad token"}}}}}}
#2020-03-2904:47yendaI would like the errors in mutations to be returned the same way as the resolvers#2020-03-3115:59wilkerlucio@yenda they are different, mutations return errors in place (as they are always flat and at the root) while the other errors are separated, if you want to change you can write a plugin with a {::p/wrap-parser ...} so you read the mutations (easy to detect: symbols at the response root) and more then to the errors map, makes sense?#2020-03-3117:25cjmurphyIf a mutation depends on inputs, that in turn depends on inputs etc, then the mutation is no longer at the root. Inputs to mutations are considered rare, and doubtless would not go very deep, but I seem to want to use them all the time.#2020-03-3121:33wilkerlucioyou can do that without changing the structure, something that I find interesting is to use mutation params as a base and then resolve to the required params, for example, lets say your mutation requires some :user/name, but the user calls the mutation with (do-something {:user/id}), pathom can figure the name from that#2020-03-3121:34wilkerluciodoing it this way it keeps it flat and on the root, and at the same time gives more freedom to the caller (doesn't have to send the exact mutation requirements, as long as there are resolvers available to get there)#2020-04-0102:03cjmurphySo inputs you send to mutations go through resolvers. Or looked at the other way params to mutations have been through resolvers already, so if there's a way (using Pathom indexes) to go from :user/name to :user/id and you send a :user/name to a mutation that requires a :user/id as param then all will be fine. (I reversed your example as I would tend to have a name in the UI during development).#2020-04-0113:27souenzzoHello pc/reader3 is trying to "inspect" my "entity" values, and it cause issues
(let [register [(pc/resolver
                  `foo
                  {::pc/input  #{:db}
                   ::pc/output [:b]}
                  (constantly {}))]
      parser (p/parser {::p/plugins [(pc/connect-plugin {::pc/register register})]})
      env {::p/reader [p/map-reader
                       pc/reader3
                       pc/open-ident-reader]}
      conn (d/connect (doto (str "datomic:mem://" (d/squuid))
                        d/create-database))
      db (d/db conn)]
  (parser env `[(:b {:pathom/context {:db ~db}})]))
reader3 will throw because "walks" into db and try to get :children from datomic....Datum reader2 work fine
#2020-04-0115:48wilkerlucio:pathom/context is only supported in ident joins#2020-04-0116:22souenzzoalso occur in ident join I removed the ident join when i'm "trim down" to the "root problem"#2020-04-0213:00souenzzobump šŸ˜ž#2020-04-0213:02souenzzo
(let [register [(pc/resolver
                  `foo
                  {::pc/input  #{:db :a}
                   ::pc/output [:b]}
                  (constantly {}))]
      parser (p/parser {::p/plugins [(pc/connect-plugin {::pc/register register})]})
      env {::p/reader [p/map-reader
                       pc/reader3
                       pc/open-ident-reader]}
      conn (d/connect (doto (str "datomic:mem://" (d/squuid))
                        d/create-database))
      db (d/db conn)]
  (parser env `[{([:a "a"] {:pathom/context {:db ~db}})
                 [:b]}]))
#2020-04-0213:08wilkerluciosorry man, I don’t have the time to dig into this now (just looking in this example I don’t have any ideas on what is wrong), too many things happening right now, if you think that’s a bug please make a full example and open an issue on pathom or pathom-datomic (wherever you find its more relevant)#2020-04-0213:11souenzzobut it should work, right?#2020-04-0213:30wilkerluciowell, I don’t see you using the datomic plugin, I dont understand what p/reader3 trying to inspect means, so I have no idea whats going on in this#2020-04-0221:30souenzzoMy problem: (map? db) return true with datomic db so functions like p/lift-placeholders that use clojure.walk, walk in datomic db, that can be problematic for performance but in my case, it's problematic because it thows when try to "walk" into Datum for examepl After fix lift-placeholders I found out that p/map->shape-descriptor has the same problem (not by clojure.walk, but by trying to use datomic db as map? ) I have 2 proposals: A: Create a IFinalValue protocol
(defprotocol IValue
  (final-value? [this]))
(extend-protocol IValue
  Object
  (final-value? [this ] false)
  nil
  (final-value? [this ] false))
So the user can extend it to datomic db for exemple to prevent this problem B: Create a ::p/final-value? optinal key inside env that defaults to (constantly false)
#2020-04-0114:48lilactownI’m trying to map out how a pathom/eql client would cache results of queries#2020-04-0114:54lilactownit seems like there are two ways to assert whether some data is about the same entity based on a result: • an ident is directly queried for e.g. [{[:customer/id 123] [:customer/name :customer/email]}] the result of which would look like {[:customer/id 123] {:customer/name "Foo" :customer/email "bar"}} • a query results in a record(s) that contain some field that identifies the entity, e.g. the query [::latest-product] that results in {:product/id 1 ,,,} #2020-04-0114:55lilactownany others people can think of?#2020-04-0114:58lilactownthe client caching problem seems a little hard to start due to the fact that there’s no explicit schema. I need some way of detecting whether something is an ident/identifying field, or have the user specify out of band. also the fact that it’s ambiguous whether a result will be a collection or scalar makes it harder to wrap my head around right now#2020-04-0115:22lilactownhmm yeah, in GraphQL the result typically has some metadata: a __typename field that denotes whether a part of a result is a queryable object. if it’s not there, then you can reason that the part of the result you’re looking at is just some random JSON, not a part of the graph#2020-04-0115:23lilactownapollo uses the typename and (by default) the id field to construct a cache key
#2020-04-0115:31lilactownyeah actually I have no idea how to tell if a part of a result is a queryable thing or just some random EDN.#2020-04-0115:47wilkerlucio@lilactown yeah, in fulcro the idents take care of this problem, because of the open nature of Pathom you need to add something like that to compare identities#2020-04-0115:47wilkerluciobecause in pathom you may have multiple identities for the same thing, and also, different identities can partially share some values, but not others#2020-04-0115:54lilactownI’m still trying to grok how idents work in fulcro (and how it’s different than idents in EQL and Pathom)#2020-04-0116:02Bjƶrn EbbinghausIt isnā€˜t different than eql and pathom. #2020-04-0115:58lilactownit seems like a system would usually have a relatively global notion of identity. e.g. customer-ident and product-ident would be used throughout the app, right?#2020-04-0115:59lilactownfulcro seems to couple identity to a component, which seems weird, but I am probably missing something#2020-04-0116:03wilkerlucio@lilactown fulcro integrates that because of automatic normalization, having that in the component allows fulcro to transform a response tree into a normalized database#2020-04-0116:04wilkerlucioso when new data enters the system, we can use a combination of component ident + props (ident is really just a fn applied to props) to decide where in the DB this data will go#2020-04-0116:04wilkerlucioso components with the same identity end up sharing data in the same place on the db (they get merged there), makes sense?#2020-04-0116:06lilactownI think so šŸ˜…#2020-04-0116:21lilactownI’d like to be able to have a library that works like:
(def all-customers-query
  [{:customers/all [:customer/id :customer/name :customer/email]}])

;; using helix for React components
(defnc customer-list
  []
  ;; read from the cache all of the customers, maybe go fetch them
  (let [customers (use-query all-customers-query)]
   ,,,))
and then elsewhere:
(def customer-detail-query
  '[{[:customer/id ?id] [:customer/name :customer/age :customer/email :customer/phone ,,,]})

(defnc customer-detail
  [{:keys [customer/id]}]
  ;; read from the cache this specific customer, maybe go fetch it
  (let [customers (use-query customer-detail-query {:id id})]
    ,,,))
and it sounds like in order for these two components to maintain data consistency, I need to also add an additional mechanism that takes the result and computes an ident for each of the queries. something like
(def customer-detail-query
  '[{[:customer/id ?id] [:customer/name :customer/age :customer/email :customer/phone ,,,]}))

(defn customer-detail->ident
  [detail]
  ;; result will be {[:customer/id ?id] ,,,}
  (first (keys detail)))

(def all-customers-query
  [{:customers/all [:customer/id :customer/name :customer/email]}])

(defn all-customers->idents
  [all-customers]
  ;; result will be [{:customer/id ?id ,,,} ,,,]
  (map :customer/id all-customers))
#2020-04-0116:22lilactownsorry for the wall of text.#2020-04-0116:24lilactownwhich I guess makes sense… but seems tedious#2020-04-0116:28souenzzoI don't think that this ?id template thing is a good path to solution The point of sale of EQL is that it's clojure data, so you don't need "query parameters" or "templates", you can just "assoc/concat" things
(def customer-detail-query
  [:customer/name :customer/age :customer/email :customer/phone])
(defnc customer-detail
       [{:keys [customer/id]}]
       ;; read from the cache this specific customer, maybe go fetch it
       (let [customers (use-query [:customer/id id] customer-detail-query)]))
Then use-query wil build/compose the final query
#2020-04-0116:29lilactownfair, it’s all just ideation#2020-04-0116:40souenzzoAs #fulcro, you can add metadata in queries
(def product-query
  ^{:ident :product/id} [:product/id,,,])
(def customer-query
  ^{:ident :customer/id} [:customer/id,,,
                          {:customer/products product-query}])
#2020-04-0116:42souenzzoBut I think other solutions also may be possible: - server response with normalization metadata - global "index key" registry - Every map should contain a :index-by :costumer/id key#2020-04-0116:44souenzzooops, reponses in wrong thread. but ok.#2020-04-0116:45lilactownhaha šŸ™‚ I appreciate the responses!#2020-04-0116:48souenzzoin fulcro case, you declare with with render so
(defsc Foo [this prop]
  {:query [:foo/id :foo/name]
  :ident :foo/id}
  (div "Ok"))
(defsc Bar [this prop]
  {:query [:bar/id {:bar/foo (get-query Foo)}]
  :ident :bar/id}
  (div "Ok"))
All query/metadata is handled my defsc and get-query
#2020-04-0116:49lilactownhow does fulcro handle multiple idents in a single query?#2020-04-0116:50lilactownor is that even a thing#2020-04-0116:57souenzzoYou can normalize the same data (users for example) both with user/id and user/email, for example I think that it's a anti-pattern. Not sure. This decision is on the component#2020-04-0116:57lilactownah, I mean like https://clojurians.slack.com/archives/C87NB2CFN/p1585758772067900#2020-04-0117:01souenzzohttps://github.com/fulcrologic/fulcro/blob/master/src/main/com/fulcrologic/fulcro/algorithms/normalize.cljc#2020-04-0117:01lilactownhahahaha fair šŸ˜‚#2020-04-0117:02souenzzo"just functions"#2020-04-0116:32lilactownthat sort of dovetails into something I’ve been wondering about: how does fulcro handle a query with multiple idents? e.g. I need the customer and the product they most recently bought:
[{[:customer/id 123]
  [:customer/name
   :customer/email
   {:customer/last-purchase
    [:product/id :product/name :product/price]}]}]
ostensibly there’s two identities here: :product/id and :customer/id
#2020-04-0219:15Alex Hit's attached to the query structure itself - so the [:product/id ...] will be tagged with metadata that it's a product#2020-04-0219:15Alex HI've built the same thing a year or so ago in re-frame and works just fine that way#2020-04-0219:16Alex Hthe metadata doesn't get passed through; it's just held in client state and picked up as the data comes back - then you have all the information you need for normalization#2020-04-0219:19Alex Hsimple example with all the sub-queries spelled out instead of hierarchically derived (have the latter, too):
(re-frame/reg-event-fx
  ::start
  interceptors
  (fn-traced [{:keys [db]} [_ code]]
    {:db db/default-db
     ::dfr/fetch {:remote :default
                  :target {:op     :replace
                           :target [:ui/set :set]}
                  :marker [:ui/set :loading?]
                  :query [{[:set/code code]
                           [:set/id :set/code :set/name :set/type :block/code
                            :block/name :set/card-count :set/released-at :set/icon-svg
                            {:set/prints
                             (df/query entity/Print
                                       [:print/id
                                        :print/collector-number
                                        :gatherer/uri
                                        :gatherer/image-uri
                                        {:>/card (df/query entity/Card
                                                           [:card/id
                                                            :card/name
                                                            :card/color-identity
                                                            :scryfall/image-uris
                                                            {:card/inventory (df/query entity/Inventory
                                                                                       [:inventory/qty
                                                                                        :print/id])}
                                                            {:card/faces [:card.face/id
                                                                          :card.face/name
                                                                          :card/supertypes
                                                                          :card/types
                                                                          :card/type-line
                                                                          :gatherer/image-uri
                                                                          :card/oracle-text]}])}
                                        {:>/inventory (df/query entity/Inventory
                                                                [:print/id
                                                                 :inventory/qty])}
                                        ])}]}]}}))
#2020-04-0219:20Alex HThe only thing df/query does there is (with-meta ...)#2020-04-0219:22Alex Hwhen the data comes back, the normalization engine just checks the metadata attached to the original query, and normalizes accordingly#2020-04-0219:25Alex HI think that's also pretty similar to what fulcro does internally#2020-04-0219:50lilactownthanks, that helps#2020-04-0219:50lilactownso it looks like you annotate each level of query with an ā€œentityā€ that allows you to associate the ident info#2020-04-0219:50Alex Hyea, pretty much - the stuff I want normalized gets annotated#2020-04-0219:53Alex Heach "entity" is just a map, effectively, with info on how to normalize (where to in the database, and which field is the ID), as well as a list of all query-able fields, for convenience, if you just want to grab everything#2020-04-0220:55lilactownhave you attempted to integrate anything with spec yet?#2020-04-0220:59Alex Hnot on the query-side, no; no plans to do so either (not obvious to me where the benefit is?); I do use spec for validation of mutations, though#2020-04-0221:04lilactownjust checking#2020-04-0221:05lilactownwe have a lot of specs right now of all our domain entities. it feels like a great extension to those to specify some normalization behavior#2020-04-0221:05lilactownit makes sense why you wouldn’t care as much if you already had these descriptions with normalization without specs. I just already have them, and don’t want to write the same thing twice šŸ˜›#2020-04-0221:07Alex Hfair#2020-04-0223:04Frank Henard#2020-04-0223:05Frank Henard#2020-04-0314:49myguidingstaris there any built-in plugin or configuration to convert all ::p/not-found values of maps in a parser's output to nil?#2020-04-0315:24kszabohttps://github.com/wilkerlucio/pathom/commit/fed5f103dcf5a63df270e02b05d0429e50595dad#2020-04-0315:25kszabonot exactly the same, it removes the keys from the response completely#2020-04-0315:25kszabobut it makes your response size smaller#2020-04-0315:25kszaboI think this should be the default or should be documented more prominently#2020-04-0315:25kszabofor Connect parsers at least#2020-04-0315:37myguidingstarexactly what I want. Thanks#2020-04-0317:38souenzzocan I say that p/reader is a "pipeline" of state-full functions that operates on env (mostly swap! on entity ) and returning a "command" that say if p/parser should continue or not the pipeline?#2020-04-0320:27wilkerluciofeels like meshing too many things, the parser has the reader pipeline, which is an internal helper, that one is the one that may have a vector of readers, and does the coordination in case ::p/continue is returned, but I consider it more lower level than the parser itself#2020-04-0320:28wilkerlucioreader is something that will process an entry of the EQL query#2020-04-0401:09russellIs it not possible to do the "fulcro client queries graphql remote directly" trick made famous by https://github.com/codonnell/crudless-todomvc with fulcro 3? It seems like pathom requires e.g. fulcro.client.network which does not seem to exist?#2020-04-0401:11russellDo I probably want a separate pathom parser server anyway?#2020-04-0401:17Chris O’DonnellIt's very possible to do it with fulcro 3.#2020-04-0401:19Chris O’DonnellThe code that implements the fulcro remote is pretty small; you can implement it yourself. I think someone posted a gist here recently showing how to do it.#2020-04-0401:21Chris O’DonnellI do personally prefer using pathom in the server now, but there are pros and cons to both approaches.#2020-04-0401:27russellIs this exactly what is meant by the simple/advanced dichotomy at https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/graphql/fulcro.html#Connect ?#2020-04-0401:28Chris O’DonnellNo, those are both talking about using pathom in the client.#2020-04-0401:31Chris O’DonnellI am writing a blog series showing how to set up an application with fulcro in the client and pathom in the server at https://chrisodonnell.dev. Another example of that setup is the fulcro template (https://github.com/fulcrologic/fulcro-template).#2020-04-0401:37russellDo you think that Hasura provides any added value when you are already running pathom on the server? Simply writing the resolvers' SQL directly seems to work well in the mygiftilst example, is there a point whre you think it becomes unwieldy?#2020-04-0401:41Chris O’DonnellHasura is an efficient and feature-rich graph query parser, and it just works without you having to write any code, which is awesome. However, when you use Hasura your query schema is forced to mirror your database table structure. In practice I've found that coupling to be pretty constraining, and so I've preferred to write the resolvers myself.#2020-04-0401:42Chris O’DonnellAnother nice advantage of hasura is its out-of-the-box support for subscriptions.#2020-04-0401:42russellYes I was primarily enticed by the possibility for it to work with ~0 glue and i was prepared to tolerate potential issues like mutations being weird#2020-04-0401:48russellIf the zero-code dream can't be realized then it does seem better to just write resolvers to SQL myself#2020-04-0401:52Chris O’DonnellIt should be possible to set up hasura without much glue#2020-04-0501:18souenzzoI'm working on a pathonized hasura šŸ˜‰#2020-04-0501:23Chris O’DonnellNeat. šŸ™‚ Is it open source?#2020-04-0518:16souenzzoNothing public ATM In our current project, we are generating resolvers from others specs like #datomic (schema), hodur, #clojure-spec. It's still binded in many other system components. But once I release the main product, I will work to split out this.#2020-04-0518:17souenzzo(swagger, SQL, GQL and other EQL API's will eventually be demand by the project)#2020-04-0518:17Chris O’DonnellCool, I look forward to it šŸ‘#2020-04-0518:17Chris O’DonnellHow are you managing authorization rules?#2020-04-0518:22souenzzoIt's still a good question. We are B2B and we kind of "don't care" to much ATM. There is 2 implemented methods: 1- "cleanup query", that given user creds, we remove some attributes/reject the query 2- There is also many instances of "parser", some parser's have more resolver then others And some "implicit" ACL's - If user don't have permissions, the 3party data source will reject - datomic(eventually SQL) has some ACL support#2020-04-0518:24souenzzoTake a look at this: https://github.com/souenzzo/eql-as#real-world-exmaple It's from export-parser-as-rest module šŸ™‚#2020-04-0518:31Chris O’DonnellšŸ‘#2020-04-0623:43Reily SiegelIf you are using Postgres, the zero-code dream can be realised with https://github.com/ReilySiegel/EQLizr (disclaimer, author). If you arent using Postgres, it should be fairly straightforward to extend, as only ANSI features are used, and can be extended to other dbs via multimethod. Note, this library currently does not support mutation.#2020-04-0401:50Chris O’DonnellThe fulcro 3 remote should look something like this (untested):
(defn fulcro-remote [parser]
  {:transmit! (fn [_ {:com.fulcrologic.fulcro.algorithms.tx-processing/keys [ast result-handler] :as send-node}]
                (let [edn (eql/ast->query ast)]
                  (go
                    (try
                      (result-handler {:transaction edn
                                       :body (<?maybe (parser {} edn))
                                       :status-code 200})
                      (catch :default e
                        (js/console.error "Pathom remote error:" e)
                        (result-handler {:body e
                                         :status-code 500}))))))})
#2020-04-0820:35λustin f(n)How can I debug the query planner, and what kinds of things confuse it? I am starting to get the query planner taking the vast majority of the query time#2020-04-0821:43wilkerlucioquery planner you mean using reader3? or the tracer?#2020-04-0822:39λustin f(n)Not sure, but I am trying to debug a long query, so I am requesting a :com.wsscode.pathom/trace along with my request. I am using the following setup#2020-04-0822:39λustin f(n)
(def server-parser
  (p/parallel-parser
    {::p/env {::p/reader [p/map-reader
                          pc/parallel-reader
                          validate-connect-key-reader
                          pc/open-ident-reader
                          p/env-placeholder-reader]
              ::p/placeholder-prefixes #{">"}}
     ::p/mutate pc/mutate-async
     ::p/plugins [(pc/connect-plugin {::pc/register (concat res-i/resolvers engine/resolvers sponsor/resolvers)})
                  p/error-handler-plugin
                  p/trace-plugin]}))
#2020-04-0822:39Ī»ustin f(n)And the 'compute-plan' step is coming back as nearly 30 seconds for my particular query#2020-04-0822:45wilkerluciook, the first guess would be too many edges, 30 seconds is really long time, are you in a case where there are a lot of mutiple input requirements on the resolvers?#2020-04-0822:58Ī»ustin f(n)So, is it expected that things will start taking a long time if my graph starts sprawling too much?#2020-04-0822:58Ī»ustin f(n)And yes, there are probably 4+ places in this query where it uses a multiple-input resolver#2020-04-0901:42wilkerlucioyeah, the planner on reader2 is quite inefficient around multiple inputs, because it does a cartesian product of branches, that means the number of generated paths multiply quickly when you have many multiple inputs combined#2020-04-0901:42wilkerluciothat said, I believe (still need to measure in the real world) that the new planner handles those much better#2020-04-0901:43wilkerlucioI suggest you try the alpha releases, and use serial-parser with reader3 (instead of parallel-reader)#2020-04-0901:43wilkerlucioand please let me know how that goes, I expect the plan computation to be considerably faster#2020-04-0921:45Ī»ustin f(n)It simply gave me a quick 'not-found' instead#2020-04-1020:31wilkerlucioI would be interest to see a reproduction of that, can you simple a demo failing case?#2020-04-0908:43timeyyyHey all. Is there a way to enable hot code reloading of pathom resolvers and mutations? I have to keep restarting my repl to get changes to my resolvers to pick up :thinking_face:#2020-04-0912:23fjolne@U7V6UECH1 you’d probably want to use some state management lib (like https://github.com/tolitius/mount) and reloaded workflow (via https://github.com/clojure/tools.namespace) then you can reload the code and reinstantiate pathom instance with one function you can look at how it’s implemented here https://github.com/fulcrologic/fulcro-template (no need to use fulcro itself)#2020-04-0913:13stevejeval the resolver and the parser again will get changes to show up. I am just experimenting with pathom in a single file to learn it and that is working for me#2020-04-0913:52fjolnethat’s a good approach for a simple setup as pathom per se doesn’t have a lifecycle semantics, but in general case there’s a http server and other stuff which depends on pathom and which needs to be properly reloaded when resolvers change#2020-04-1018:18kszaboI would recommend integrant: https://github.com/weavejester/integrant after using 4 years of mount#2020-04-1018:18kszaboDI > globals#2020-04-1018:18kszabowhich mount promotes#2020-04-1019:52fjolne@U08E8UGF7 i also used integrant on a few commercial projects, but one state map + multi-methods all over the place turned out to be somehow less manageable i’d really appreciate if you explained a bit more why you prefer integrant i might guess that’s in line with reasoning why people prefer data-driven vs macros like in the the shift from compojure to reitit i also see how integrant is more manageable in terms of testing, but still multi-methods #2020-04-1020:09kszaboit’s exactly your points. Multi-methods is a necessary evil that I’m willing to take on. (I would prefer the data-driven pathom resolver approach if I would design it today). One benefit though with the ns-load side-effects that you can use the built-in load-namespaces function and name your components with keywords that have backing namespaces, then you can start your system map in your cli namespace without any knowledge where those multimethods reside. Essentially the system map can always initialize itself by deriving the implementation from it#2020-04-1020:21fjolnehuh, seems like i’ve been doing a similar thing, but in a more hacky way and felt guilty for it, thanks! might be worth giving integrant another spin#2020-04-1112:33kwladykaIs it fine to use forĀ `:pc/input`Ā vector instead ofĀ `#{}`? Or on the other hand forĀ `:pc/output`Ā useĀ `#{}`Ā instead of vector, which will be even more useful for me. It looks like it works, but do I miss something? There has to be some reason when it is done in that way in doc right?#2020-04-1112:40souenzzo
(s/describe ::pc/input)
=> (coll-of :com.wsscode.pathom.core/attribute :kind set?)
(s/describe ::pc/output)
=>
(or
 :attribute-list
 (coll-of :com.wsscode.pathom.connect/out-attribute :kind vector? :min-count 1)
 :union
 (map-of :com.wsscode.pathom.connect/attribute :com.wsscode.pathom.connect/output))
It's values are specified
#2020-04-1112:41kwladykaah I see. I had to close and run REPL again to crash evrything#2020-04-1112:43kwladykaDo you see this patterns also for your app when you have to use:
(def shop-keys [:shop/uuid :shop/name :shop/engine :shop/config])
without uuid [:shop/name :shop/engine :shop/config] as a vector and set?
#2020-04-1112:43kwladykaso on the end it is a little redundant#2020-04-1112:45kwladykaas long as it is macro anyway I would like to see this converting to the right type (set / vector)#2020-04-1112:45kwladykabut maybe it is only my case#2020-04-1113:00kwladyka
(def shop-keys-without-uuid #{:shop/name :shop/engine :shop/config})
(def shop-keys (into [:shop/uuid] shop-keys-without-uuid))

(defn ->spec [shop]
  (s-utils/map-coercion
    {:shop/uuid uuid/as-uuid
     :shop/config json/read-str}
    shop))

(defn ->eql [shop]
  (-> (update shop :shop/uuid str)
      (dissoc :shop/created_at :shop/updated_at)))

(pc/defmutation create-shop! [env shop]
  {::pc/sym 'shop/create!
   ::pc/params shop-keys-without-uuid
   ::pc/output shop-keys}
  (->eql (shop-db/create-shop! (->spec shop))))

(pc/defmutation update-shop! [env shop]
  {::pc/sym 'shop/update!
   ::pc/params (set shop-keys)
   ::pc/output [:shop/uuid]}
  (shop-db/update-shop! (->spec shop))
  (select-keys shop [:shop/uuid]))

(pc/defresolver get-shop [env params]
  {::pc/input #{:shop/uuid}
   ::pc/output (vec shop-keys-without-uuid)
   ::pc/transform pc/transform-batch-resolver}
  (shop-db/get-shops-by-uuid (->> (map :shop/uuid params)
                                  (map (partial s-utils/safe-coercion uuid/as-uuid))
                                  (set))))

(pc/defresolver all-shops [env _]
  {::pc/output [{:shops shop-keys}]}
  {:shops (mapv ->eql (shop-db/get-all-shops))})

(pc/defmutation delete-shop [env {:shop/keys [uuid]}]
  {::pc/sym 'shop/delete
   ::pc/params #{:shop/uuid}
   ::pc/output [:deleted?]}
  {:deleted? (shop-db/delete-shop uuid)})
This dance with set and vec and how def looks show what I mean.
#2020-04-1114:50pithylessIn your case, create-shop!, get-shop, and all-shops all return [:shop/name :shop/engine :shop/config]. I find this approach counter-intuitive, as subtle changes in implementation of these resolvers could hide bugs over time. I prefer to have create-shop!, all-shops , and update-shop! all return just :shop/uuid; and then you only need one get-shop that knows how to resolve all the other attributes. Easier to isolate logic/authorization/etc. to one place and you no longer have to worry about code redundancy. ;]#2020-04-1115:19kwladykahmm while database performance and time response is not the issue, because of the 2x DB query it is a solution šŸ™‚#2020-04-1115:20kwladykathanks#2020-04-1119:15myguidingstar@wilkerlucio regarding dynamic resolvers, do all items in ::pcp/requires have equivalence in ::pcp/foreign-ast and vice versa? Also, what is the use case for those ::pcp/requires?#2020-04-1215:04wilkerluciopcp/requires determines what is expected to be on the output of this node call#2020-04-1215:05wilkerluciofor dynamic, yes, it should have the equivalent in foreign ast#2020-04-1320:15kwladykahow do you use pathom with PHP?#2020-04-1320:15kwladykaI found https://github.com/igorw/edn but I doubt it is compatible with clojure EDN#2020-04-1320:15kwladykais it?#2020-04-1320:15eoliphantpathom is clojure/script only#2020-04-1320:16kwladykaYes, but probably I will have to use it also in other languages. At least some part of API#2020-04-1320:16kwladykajust thinking if it is possible to achieve#2020-04-1320:16eoliphantyou could maybe create rest/graphql apiis that delegate to pathom#2020-04-1320:16kwladykawith good EDN converter it should be#2020-04-1320:17eoliphantit’s possible, may not be practical#2020-04-1320:17kwladykaI would really like to avoid maintenance 2 APIs doing the same šŸ˜•#2020-04-1320:18kwladykaIs some kind of magic to interage pathom with https://github.com/walmartlabs/lacinia by automate without rewriting the code?#2020-04-1320:19eoliphantthere’s a lib, that does that#2020-04-1320:19kwladykahmm and does it work without any issues?#2020-04-1320:20eoliphanthttps://github.com/denisidoro/graffiti#2020-04-1320:20eoliphantnot sure, but give it whirl#2020-04-1320:21eoliphantone issue, is that it wants you to define stuff via it’s api#2020-04-1320:21eoliphantso not sure how useful it is for say ā€˜adaptiing’ an existing pathom api#2020-04-1320:22kwladykadamn… I don’t like my issue šŸ˜‰#2020-04-1320:23kwladykaah live would be so easy if other languages will use EDN too#2020-04-1320:23kwladykathis is so great data format#2020-04-1320:50kwladykaprobably I will end with pathom + REST :thinking_face:#2020-04-1320:51kwladykawill see#2020-04-1417:34eoliphantone thing to consider is that EQL/Pathom, GraphQL, Falcor etc are ways to get to more expressive apis, than what strict ā€œRESTā€ allows. So you might be able to do something where you’re using some sort of determinstic mapping from HTTP/JSON to Pathom. Somethnig like POST /mutation/ns.createSometing , POST /query etc#2020-04-1716:38wilkerluciohello folks, I’m pleased to announce that Pathom finally has a logo! The repo is updated to show it and more things are on the way, stay tuned! pathom#2020-04-1718:30myguidingstar@wilkerlucio now that I understand planner's foreign-ast, does it make sense to make use of core join (`p/join`) to process recursively the ast? maybe in a separate parser and calling that from the dynamic resolver#2020-04-1718:31myguidingstaror is there any other facility provided by pathom helping with that that I don't know of#2020-04-1722:43wilkerlucio@myguidingstar I highly recommend you don’t use p/join in the middle of your processing, the planner is doing the work to send you only the AST you can answer, and nothing else, so you should process that directly, this is more like what you did on previous walkable version, its ok to use a secondary parser if you want to internally, but using p/join will trigger all the user extensions as well, and I believe this will generate more overhead than it should#2020-04-1807:07myguidingstarright, I didn't think of the overhead of p/join#2020-04-1809:56myguidingstarafter some thought I guess I'll make use of clojure.zip or clojure.walk to recursively process the AST instead of using an secondary parser#2020-04-1904:38Reily SiegelThought I should cross-post this here, as it uses pathom. In addition to now supporting google sheets, this library supports generating Connect resolvers for PostgreSQL with almost zero configuration (Just a JDBC connectable). https://clojurians.slack.com/archives/C06MAR553/p1587271017250600#2020-04-1907:18myguidingstarvery nice @reilysiegel#2020-04-1914:59Jakub Holý (HolyJak)Hi! Is https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/introduction.html available as a single-page document somewhere? (So that I could get it into my Kindle.) thank you!#2020-04-2008:00kwladykayep, searching in doc is a little pain#2020-04-2008:01kwladykabut I don’t think we have other option now#2020-04-2110:08jeroenvandijkHi, I’m exploring the features of Pathom. I’m wondering is there a builtin way to merge independent queries during one request. E.g. I would have an html page with independent blocks.
[:div
 [:div {:class "block 1"}
   (q [:album/name
       {:album/artist
        [:artist/name]}])]
 [:div {:class "block 2"}
   (q [:album/name
       {:album/tracks
        [:track/name]}])]]
Ideally, I would render this page after collecting all queries, and do only one query to the server:
[:album/name
 {:album/tracks
  [:track/name]
  :album/artist
  [:artist/name]}]
After receiving the result of the server it could be split into seperate results for the seperate queries. Does such an in-memory resolver already exist?
#2020-04-2110:10kszaboYes, Fulcro does this automatically#2020-04-2110:11kszabohttp://book.fulcrologic.com/#_server_interaction_order#2020-04-2110:13jeroenvandijkThanks, good to know. But is not part of Panthom? So I have to steal that there. (i’m not using Fulcro)#2020-04-2110:23jeroenvandijkMaybe I’m looking for the Request cache (mentioned in the introduction). I’ll have a read
Request cache: For caching the results of parsing repetition that can happen on a single request.
#2020-04-2110:32kszabothose are separate ideas#2020-04-2110:33kszaboyou can easily merge separate EQL queries into one (if they do not have conflicts)#2020-04-2110:33kszabothen that single EQL query resolution can be made more efficient inside Pathom using the request cache (and other techniques)#2020-04-2110:34kszabohttps://github.com/edn-query-language/eql/blob/master/src/edn_query_language/core.cljc#L532-L536#2020-04-2110:34kszabothis is what you are looking for probably#2020-04-2110:50jeroenvandijkah thanks#2020-04-2110:51jeroenvandijkI’ll do some reading and give this a try later#2020-04-2113:58souenzzo@jeroenvandijk just "merge" your queries
(let [{:>/keys [a b]} (q [{:>/a [:album/name
                                 {:album/artist
                                  [:artist/name]}]}
                          {:>/b [:album/name
                                 {:album/tracks
                                  [:track/name]}]}])]
  [:div
   [:div {:class "block 1"}
    a]
   [:div {:class "block 2"}
    b]])
#2020-04-2120:28jeroenvandijkThank you. I’ll give it a try#2020-04-2114:03souenzzoAlso you can use ::p/entity
(let [a (atom 0)
      register [(pc/resolver
                  `a 
                  {::pc/output [:a]}
                  (fn [_ _]
                    {:a (swap! a inc)}))]
      parser (p/parser {::p/plugins [(pc/connect-plugin {::pc/register register})]})
      env {::p/reader [p/map-reader
                       pc/reader2
                       pc/open-ident-reader
                       p/env-placeholder-reader]
           ::p/placeholder-prefixes #{">"}}
      ->cache (fn [parser]
                (let [entity (atom {})]
                  (fn [tx]
                    (parser (assoc env ::p/entity entity)
                            tx))))
      single-use-cached-parser (->cache parser)]
  [:without-cache [:div 
                   (parser env [:a])
                   (parser env [:a])]
   :with-cache [:div
                (single-use-cached-parser [:a])
                (single-use-cached-parser [:a])]])
#2020-04-2205:58jmayaalvAfter following the instructions to build Pathom-Viz i get this when running the app: main.js:1 Failed to load resource: net::ERR_FILE_NOT_FOUND#2020-04-2205:59jmayaalvthis is on macos. kind of clueless what am i doing wrong, any pointers?#2020-04-2210:50Jakub Holý (HolyJak)Hello! What is the difference between https://wilkerlucio.github.io/pathom/ and https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/ (except the look & feel)? The former seems also to cover v2.2.0 based on the fact it has the section "https://wilkerlucio.github.io/pathom/#_2_2_0_upgrade_guide"? I would like to convert it into PDF for reading on my Kindle and thus the single-page /pathom/ is much better than the multipage /v2/. Thanks!#2020-04-2213:24Jakub Holý (HolyJak)I am looking at https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/graphql/fulcro.html#_setting_up_connect_with_graphql, do I understand correctly that the only place that binds the defined GraphQL endpoint (`github-gql`) is when it is used to load indexes (https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/graphql/fulcro.html#_loading_the_graphql_schema_and_creating_a_remote)? I guess Pathom Connect then has all info necessary in the indexes, so it can call the GQL endpoint when necessary? And, BTW, is it possible to configure the endpoint more? In particular I need to include a header with an API token. I guess I just add ::p.http/headers {"x-api-key" "123"} to the endpoint's def , right? But what if the header is dynamic and depends on some other stuff from the environment? (For some queries, I also need to include the id of the user making the request; but it is expected to be provided as a header, not a query param.)#2020-04-2213:50Chris O’Donnell@holyjak Your understanding is correct about loading indices. I've added a bearer token dynamically to requests using a parser plugin like this:
(def auth-plugin
  {::p/wrap-parser
   (fn [parser]
     (fn [env tx]
       (parser
         (assoc-in env [::p.http/headers :authorization] (str "Bearer " (get-token)))
         tx)))})
#2020-04-2214:32Jakub Holý (HolyJak)Awesome! Thanks a lot!#2020-04-2215:48jmayaalvhello, totally newbie with pathom here. i am trying to setup worskpaces with a pathom-viz card. is there any example/doc to follow when pathom is running on clj? example here: https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/exploration.html passes a parser, but how to pass a parser when pathom is running on clj and not cljs?#2020-04-2316:00jmayaalvfound what i needed on https://github.com/jlesquembre/pathom-pedestal šŸ™‚#2020-04-2216:07jmayaalvam i correct assuming that it should be something similar to standalone ?#2020-04-2216:07jmayaalvhttps://github.com/wilkerlucio/pathom-viz/blob/b603b0c021a954fa6cd73293618bdac8154ee5d8/src/core/com/wsscode/pathom/viz/standalone.cljs#2020-04-2311:23Jakub Holý (HolyJak)Anyone experienced with Pathom Connect and GraphQL integration? Is there a way to teach it to understand errors (since GQL always return 200), similar to Fulcro's configurable :remote-error? ? I know that if the response boody contains :data nil, :errors [..] something went wrong and I want to get and e.g. log the errors. Hm, I see that graphql-resolve actually extracts both data errors from the response, just doesn't show the errors to me... Do I need to add something to the query or add some pathom plugin to be able to see the graphql errors? I have added these to my query :com.wsscode.pathom.connect.graphql2/errors :com.wsscode.pathom/trace but I got nothing back.#2020-04-2312:53Chris O’Donnell@holyjak You can customize the parser that gets called to inject the errors as you like into your result. Here's an example parser: https://wilkerlucio.github.io/pathom/#CustomResultParsing. Looking at graphql-resolve, the errors should be available under ::pcg/errors in env. I have not done this personally, but I believe it should work looking at the code.#2020-04-2415:00Jakub Holý (HolyJak)Hi! If you learn something about the graphql integration, especially ::pcg/ident-map and queries with fields that take 1 or more params, share it! I really struggle here... (The docs worked for me for a top-level field but I struggle for nested ones.)#2020-04-2419:56Chris O’DonnellI would love to help, but just don't have the time at the moment!#2020-04-2313:24Jakub Holý (HolyJak)Thanks! Looking at com.wsscode.pathom.connect.graphql2/error-stamper it seems it should actually propagate these errors to the ::p/errors* , I wander why it does not work for me... Thanks for the tip, this might help in troubleshooting!#2020-04-2313:35Jakub Holý (HolyJak)I found it, error-stamper ignores the error because its path does not match with what it expects. Opened an issue: https://github.com/wilkerlucio/pathom/issues/152#2020-04-2412:18jmayaalvhello, wanted to say that we have been evaluating pathom last few days and we are blown away. Serioulsy good job @wilkerlucio#2020-04-2412:18jmayaalvand commiters#2020-04-2412:18jmayaalvthank you for such a great tool#2020-04-2412:19jmayaalv(cant’ go back to graphql now) šŸ˜ž#2020-04-2414:50myguidingstar@jmayaalv it would be great if you can share your evaluation#2020-04-2415:53Jakub Holý (HolyJak)Hello! I struggle translating a working GraphQL query into Pathom Connect + graphql. I have described my problems here https://github.com/wilkerlucio/pathom/issues/155 and would appreciate any help. Thank you!!!#2020-04-2417:45wilkerluciothanks for the great report, I'll have some time to look into it later today#2020-04-2418:40Jakub Holý (HolyJak)Thanks a lot! I plan to add a little more high level tests for the GraphQL PC that could also serve as a documentation (query + index -> resolver -> driver that returns hard-coded response -> errors and data). The current unit tests are too low for that.#2020-04-2420:53Jakub Holý (HolyJak)@wilkerlucio Could you perhaps be so kind and show me how to write such a high-level/integration style? I see I know too little to be able to pull it off. Something like running
(parser
  {::pc/indexes <based on graphql2-test/indexes>
   ::pcg/prefix "service"
   ::pcg/ident-map {...}
   ::p.http/driver (fn [r] (verify-expected-query r)
        "{\"data\":{...}}")}
   [{[:service.Customer/id "123"]
     [:service.Customer/cpf]}]
? I guess that graphql2-test/query-env quite close to the env I need, I would perhaps just need to plug into the env the actual com.wsscode.pathom.connect.graphql2/graphql-resolve šŸ™
#2020-04-2421:25wilkerluciofor the integrations like these, I want to warn you they are about to change, I plan to release this weekend some of the new stuff, and sadly the graphql integration will need rework (graphql3 is coming šŸ˜›, but probably not this weekend), so I like to write about those integrations for this new scenario, since this is the thing moving forward#2020-04-2600:03sergey.shvetsHi! Fantastic library, thank you @wilkerlucio! I'm stuck with collections of items and proper output string for resolver. Let's say I have a collection of items that returns comments for videos. They all have author id. How do I make the ::pc/output structure so I can join in queries on author_id? [[:comment/id :comment/text :comment/author_id]] doesn't work.#2020-04-2601:33wilkerlucioI imagine something like: ::pc/output [:comment/id :comment/text :comment/author-id]#2020-04-2601:33wilkerluciobut then you need resolvers to traverse from :comment/author-id to its data#2020-04-2601:54sergey.shvetsDo I need a special indication that I return a collection of maps and not a single map?#2020-04-2605:50wilkerlucio#2020-04-2607:43sergey.shvetsI got this error when trying to build an app with viz-connector: The required namespace "com.wsscode.async.processing" is not available, it was required by "com/wsscode/pathom/viz/ws_connector/impl/sente_cljs.cljs".#2020-04-2607:57sergey.shvetsThere is a small typo in docs: https://take.ms/dJ8DK#2020-04-2607:58sergey.shvetsTo solve problem above I had to manually add [com.wsscode/async "1.0.4"] to my shadow-cljs.edn.#2020-04-2607:58sergey.shvetsAfter that works like a charm! HUGE THANKS!#2020-04-2608:07wilkerluciothanks, fixed!#2020-04-2608:09wilkerlucioalso tip to foce async version, but I will make pathom use a more recent one as well, I guess its one conflicting on it: https://roamresearch.com/#/app/wsscode/page/fI5tcHpMk#2020-04-2616:03folconIs there a straightforward way to get a pathom resolver to call another one? (I’m using it from Fulcro if that helps?)#2020-04-2616:08myguidingstar@folcon can you give an example?#2020-04-2616:09folconEG: I have a pathom resolver:
(pc/defresolver session-account-resolver [env _]
  {::pc/output [:account/name]}
  (let [session (get-session env)
        username (some-> session :account/name)]
    {:account/name username}))
Is there some way I can just:
{::pc/input  [:account/name]}
I’ve worked out that I can do this manually by doing:
(defmutation account-resolver [{:keys [parser] :as env} _] {} {:account/name (parser env [:account/name])})
Which gives me a manytomany channel, happy to work with this directly, but just wondering if there’s a helper function to do this? Or some other way to do this?
#2020-04-2616:15myguidingstar@folcon what is the input and output of the mutation account-resolver?#2020-04-2616:17myguidingstarand how would you use it?#2020-04-2616:20myguidingstaranyway, I don't find any problem with that approach of calling parser#2020-04-2616:22folconWell what I’d like to do is this:
(pc/defresolver account-resolver [env opts]
  {::pc/input   [:account/name]
   ::pc/output [:account/info]}
  {:account/info (get-account-info (:account/name opts))})
#2020-04-2616:23folconAnd pathom works out, why yes I do know how to resolve :account/name, let me grab that and pass it in!#2020-04-3014:07dvingoWas this not working? I have it working with a similar setup:
(pc/defresolver current-user-res [env _]
  {::pc/output [:app/current-user]}
  {:app/current-user (get-current-user env)})

;; pathom resolves the current user into the input params here

(pc/defresolver user-tasks-resolver [env {:keys [:app/current-user]}]
  {::pc/input  #{:app/current-user}
   ::pc/output [{:user/tasks [:task/id :task/description]}]}
  (pull [{:user/tasks [:task/id :task/description]}] current-user))
I don't think it's supported for mutations. for that i just invoke the helper (get-current-user) directly in the mutation.
#2020-04-3014:30folconHmm, I was predominantly using it in mutations, I’ll see if there’s a difference in how they’re run…#2020-04-2616:24myguidingstarwhy is it a mutation?#2020-04-2616:25folconSorry, that’s just me using it as an example, in this case it should be a resolver… But I’d like to be able to do it for mutations too =)…#2020-04-2616:26myguidingstarmaybe you're looking for "mutation joins"? https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/connect-mutations.html#_mutation_joins#2020-04-2616:26folconThis sort of thing is also really useful for things that are behind auth, so I can check, does the resolver for a session return a value? Ok, then grab the connected data associated with that account…#2020-04-2616:26folconChecking =)…#2020-04-2616:31folconNot quite that, I’ve done that before. In this case I’m trying to get pathom to get data that it knows about, as it has a resolver for it without me having to manually add the code that gets it into the mutation/resolver… Perhaps I should just try manually calling the built in parser function =)… Thanks for the help @myguidingstar!#2020-04-2616:32myguidingstaryou're welcome#2020-04-2620:36Chris O’DonnellAnother option could be to write a plug in that injects the necessary data into the environment so your resolver can pull it out from there. Don't know enough about your use case to know if that's viable.#2020-04-2623:09folconNot really, I’m finding the solution of using the parser inline is sufficient =)… I did find a reference in the chat log that what I wanted to do is called nested inputs, apparently it’s something that may be supported in future, but not at present…#2020-04-3008:26TuomasHi a pathom+clojure newbie here trying to learn how to build fulcro+datomic web app. Despite the docs and examples I’m having a hard time why my mutation isn’t found. In the fulcro inspect query I query for
[{(app.models.session/login
   {:user/email "asd" :user/password "letmein"})
  [:user/token]}]
and get
{app.models.session/login
 {:com.wsscode.pathom.core/reader-error
  "Mutation not found - {:mutation app.models.session/login}"}}
I looked at archives and saw a tip to the check if its in the index and it seems to be there
(p/env-wrap-plugin (fn [env]
                                       (println "index-mutations" (get-in env
                                                                      [:com.wsscode.pathom.connect/indexes
                                                                       :com.wsscode.pathom.connect/index-mutations
                                                                       'app.models.session/login]))
; index-mutations {:com.wsscode.pathom.connect/sym app.models.session/login, :com.wsscode.pathom.connect/mutate #object[app.models.session$login__34847 0x347d0002 
I tried using ::pc/sym ’login and ::pc/sym ’user/login but I keep getting the same error. I’ll keep going through the docs and examples, but I don’t know what to try next.
#2020-04-3008:41fjolne@koivistoinen.tuomas IIRC there was a bug in Inspect with mutations-not-found in the Query tab. have you tried calling Pathom parser directly from the REPL?#2020-04-3008:42TuomasI haven’t. I’ll figure out how to do it and try it#2020-04-3008:51fjolneit’s actually pretty easy: parser is just a function which takes env and tx if you’re using fulcro-template then the only external thing in the env is ring/request which you can put there by hand (if you really need to i.e. if your resolvers/mutations depend on it), otherwise it’s just (parser {} [{(your-mutation {:param 1}) [:data]}])#2020-04-3010:48TuomasHuge thanks it works! I struggled a while with an invalid expression error. If everyone has the same problem, be sure to quote the tx
(async/<!! (pathom-parser {} '[{(mutation {:param "value"}) [:selection]}))
#2020-04-3011:29fjolneah yeah, you typically fully-qualify the tx with the syntax-quote and then use ~ for param symbols substitution https://clojure.org/reference/reader#2020-04-3017:27wilkerlucio@koivistoinen.tuomas also, the bug around mutation responses is fixed on the Pathom Viz standalone app, if you wanna use that: https://clojurians.slack.com/archives/C06MAR553/p1587880194322000#2020-04-3018:20Bjƶrn Ebbinghaus@wilkerlucio The Index Explorer in the latest Fulcro Inspect Binary seems to have broken CSS All classes in the style-tag have unknown_unknown__ as prefix.#2020-04-3018:20Bjƶrn Ebbinghaus
.unknown_unknown__container {
  display: flex;
  flex-direction: column;
  width: 100%;
  height: 100%;
  overflow: hidden;
}

.unknown_unknown__selector {
  font-family: sans-serif;
  user-select: none;
  align-items: center;
  padding: 12px;
  border-top: 1px solid #ccc;
  font-size: 12px;
  display: flex;
  background: #F3F3F3;
  color: #5A5A5A;
}
```
#2020-04-3018:23wilkerluciothats weird, @U0CKQ19AQ you have an idea what may caused this?#2020-04-3018:25wilkerlucioas a secondary option, you can use the standalone pathom app#2020-04-3019:35tony.kaynope#2020-04-3019:35tony.kayprobably just didn’t end up in the electron resources properly#2020-04-3019:36tony.kayoh…unknown unknown….sounds like the binding of *parent* is hosed and it is using garden#2020-05-0300:04kennyIs there a way to debug why pathom is calling a particular resolver?#2020-05-0302:04dvingoi'm not sure of the answer but this may help:#2020-05-0302:04dvingohttps://github.com/wilkerlucio/pathom-viz/#2020-05-0319:54kennyOk so I got this working. Very cool! I see it calling a nested resolver multiple times but I don't understand why. Is there docs on how to read the graph shown by pathom-viz? The use case is debugging a batch query issue where is is calling an individual resolver multiple times when I think it should not.#2020-05-0300:23MatthewDoes anyone have a good reference set up for a pathom.core/parser for a node js environment (pathom mvn/version ā€œ2.2.31ā€)? I’ve switched the pathom.connect/reader2 to async-reader2 and I started getting async channels returned from my parser, but I’m wondering if there’s more to it. Any descriptions or links to a repo are appreciated!#2020-05-0301:57kennyWhen writing derived/computed attribute resolver, does Pathom know to deeply merge the result?#2020-05-0319:37wilkerluciohello @matthew_lefebvre, did you changed the parser to async-parser?#2020-05-0319:37wilkerlucioalso, when switching to it, the parser itself will start returning a core async channel, reading on it will give you the full result#2020-05-0319:54kennyOk so I got this working. Very cool! I see it calling a nested resolver multiple times but I don't understand why. Is there docs on how to read the graph shown by pathom-viz? The use case is debugging a batch query issue where is is calling an individual resolver multiple times when I think it should not.#2020-05-0319:57kennyAre the numbers here the index of the item in a list?#2020-05-0320:01kennyEvery single index that has a horizontal bar in this screenshot is an item without a :name attribute & Pathom attempting to call an individual item resolver to find it.#2020-05-0320:00kennyI think my specific problem has to do with a resolver returning a list of items. Some of those items have a :name attribute and some don't. For the ones that don't, Pathom will call a resolver that works on the individual item (perhaps hoping it could resolve the :name). That resolver will also not return a :name because it simply does not exist on certain items. Is there a way to tell pathom that an attribute will not become available no matter how many other resolvers it calls?#2020-05-0320:14kennyIn general, how do people handle optional attributes? Do you not include them in the ::pc/output vector?#2020-05-0320:01kennyEvery single index that has a horizontal bar in this screenshot is an item without a :name attribute & Pathom attempting to call an individual item resolver to find it.#2020-05-0320:45kennyAh, I have found the bug/missing thing that I thought worked and does not. Here's a repro:
(pc/defresolver videos-resolver
  [_ _]
  {::pc/input  #{:user/id}
   ::pc/output [{:user/videos [:video/id :video/name :video/info]}]}
  {:user/videos [{:video/id   1
                  :video/name "v1"
                  :video/info {:a "a"}}]})

(pc/defresolver video-extra-info
  [_ _]
  {::pc/input  #{:video/id}
   ::pc/output [{:video/info [:b]}]}
  {:video/info {:b "b"}})

(parser
   {}
   [{[:user/id 1] [{:user/videos [:video/id
                                  {:video/info [:b]}]}]}])
=> {[:user/id 1] #:user{:videos [#:video{:id 1, :info {}}]}}
Even though my query is asking for {:video/info [:b]} and a resolver is capable of resolving that key, it is not included in the output. Is there a way around this?
#2020-05-0320:48kennyI could get hacky and do this:
(pc/defresolver video-extra-info
  [env _]
  {::pc/input  #{:video/id}
   ::pc/output [:video/duration
                {:video/info [:b]}]}
  (swap! (::p/entity env) assoc-in [:video/info :b] "b")
  {:video/duration 1
   :video/info     {:b "b"}})

(parser
 {}
 [{[:user/id 1] [{:user/videos [:video/id
                                :video/duration
                                {:video/info [:b]}]}]}])
(parser
 {}
 [{[:user/id 1] [{:user/videos [:video/id
                                :video/duration
                                {:video/info [:b]}]}]}])
As long as I have an additional key that would cause Pathom to hit the video-extra-info resolver, that works.
#2020-05-0320:51kennyTo be clear, the problem is that Pathom is not letting me update a nested map that is already partially resolved.#2020-05-0320:58dvingoi think the issue may be both resolvers have ::pc/output containing :video/info , this is working for me:
(pc/defresolver videos-resolver
  [_ _]
  {::pc/input  #{:user/id}
   ::pc/output [{:user/videos [:video/id :video/name]}]}
  {:user/videos [{:video/id 1 :video/name "v1"}]}) 

(pc/defresolver video-extra-info
  [_ _]
  {::pc/input  #{:video/id}
   ::pc/output [{:video/info [:b]}]}
  {:video/info {:b "b"}})
#2020-05-0320:59dvingo
(myparser
    {}
    [{[:user/id 1] [{:user/videos [:video/id
                                   :video/name
                                   {:video/info [:b]}]}]}])
=> {[:user/id 1] #:user{:videos [#:video{:id 1, :name "v1", :info {:b "b"}}]}}
#2020-05-0321:42kenny@danvingo But I want it the way I wrote it -- a resolver will compute some new attributes on a nested entity.#2020-05-0322:04dvingo
(pc/defresolver task-resolver [env {:keys [task/id]}]
  {::pc/input  #{:task/id}
   ::pc/output [:task/description]}
  {:task/description "task Description"})

(pc/defresolver habit-resolver [env {:keys [habit/id]}]
  {::pc/input #{:habit/id}
   ::pc/output [:habit/description {:habit/task-id [:task/id]}]}

  {:habit/description "description" :habit/task-id {:task/id "task-id"}})

(myparser {} 
[{[:habit/id :habit] [:habit/id :habit/description {:habit/task-id [:task/description]}]}])
;; =>
{[:habit/id :habit] #:habit{:id :habit, :description "description", :task-id #:task{:description "task Description"}}}
I think this is what you're looking for - you have to wrap the property you want to join in a map. I just ran into this as well.
#2020-05-0322:06kennyI'd like to not have to wrap them though. It forces out new keys that have no purpose.#2020-05-0400:35dvingoyep, it surprised me too, but not a big deal, i just add an alias:
(def habit-task-alias-res (pc/alias-resolver :habit/task-id :task/id))
#2020-05-0400:35dvingoand convert that prop to a map#2020-05-0321:45dvingoi don't understand, video-extra-info is doing that#2020-05-0321:49wilkerlucio@kenny the thign is that once a property is resolved, pathom wont try to resolve again, so you can't have a "merge" like you are thinking#2020-05-0322:05kennyOh šŸ˜ž Is there a technical reason for this?#2020-05-0407:04wilkerluciowould put a new whole dimension of processing, it goes against some basic heuristics of how the library works, the same attribute appearing multiple times is expected to be just different paths for the same data, not an accumulator thing#2020-05-0407:05wilkerluciootherwise the engine would have to always process every path, that could get a out of control quick#2020-05-0321:49wilkerluciowhen it sees :video/info present, it wont try to get more of it#2020-05-0321:49wilkerluciobut, if you had something more nested, it would work, like this:#2020-05-0321:50wilkerlucio
(pc/defresolver videos-resolver
  [_ _]
  {::pc/input  #{:user/id}
   ::pc/output [{:user/videos [:video/id :video/name]}]}
  {:user/videos [{:video/id 1 :video/name "v1"}]})

(pc/defresolver video-extra-info
  [_ _]
  {::pc/input  #{:video/id}
   ::pc/output [::video/info-b]}
  {:video/info-b "b"})
#2020-05-0321:53mruzekwHi there, if I'm using Datomic, how do I make sure that my resolver is always getting the latest value of the DB? Would I need pass down the DB connection through the context and get the value in each resolver? Or is there another way?#2020-05-0321:55eoliphantif you mean ā€˜latest’ in terms of perhaps during a mutation join, or something, you’d need to either pass in the conn and grab the db in your resolver, or write a plugin that say updates it in the env after a mutation.#2020-05-0321:56eoliphanti played around with it a bit last year#2020-05-0321:56mruzekwI mean, I'd want the latest value (`(d/db conn)`) at any time query or mutation#2020-05-0322:00lilactownYou probably want to get it once for the duration of a query#2020-05-0322:01lilactownOtherwise you’re going to end up with potential inconsistencies #2020-05-0322:01eoliphantas opposed to simply using the one that you pass in when you invoke the parser? such that if anything changes while the resolution is happening, you have the latest one? You’d still need either approach. my plugin experiment put the conn and the db in an atom, and refreshed the db after mutations. You could do the same to have it happen after anything. It gets a little weird because on the one hand, part of datomic’s value in many cases is having a ā€˜stable’ view of the database, which is really nice when you’re doing pathom/graphql/etc, but there are obvious cases, like mutation joins where you really need an update#2020-05-0322:03mruzekwI guess where I'm confused is that if I just pass (d/db conn) to the parser like so:
(def parser
  (p/parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/reader2
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}
                  :db (d/db conn)}
     ::p/mutate  pc/mutate
     ::p/plugins [(pc/connect-plugin {::pc/register my-resolvers})
                  p/error-handler-plugin
                  p/trace-plugin]}))
#2020-05-0322:03mruzekwSorry slack trouble#2020-05-0322:04mruzekwI would only get one value of the db ever, right?#2020-05-0322:04eoliphantah you’d typically pass it in#2020-05-0322:04eoliphantwhen you actually invoke the parser (parser {:db (d/db conn)} ….)#2020-05-0322:06mruzekwGot it.#2020-05-0322:07dvingoYou can use the env-wrap-plugin:
(defn augment-env-request
  [env]
  (assoc env
    :db db
    :current-user (user/get-current-user env)
    :config config))
;; add to plugins vector:
(p/env-wrap-plugin augment-env-request)
#2020-05-0322:07mruzekwThe parser is not a long-standing process, it's just a fn#2020-05-0322:07eoliphantas opposed to when you create it. Or, you could do what you’re doing there, but pass in the conn. and have a plugin assoc in the db or something. but the simplest way to get going is pass it in when you invoke it#2020-05-0322:08mruzekwOkay, I'm understanding better now#2020-05-0322:08eoliphantyep, the value of values, higher order funcs, and all that jazz šŸ™‚ It’s a function that’s closed over all your setup, and acts accordingly when you invoke it#2020-05-0322:08mruzekwCool, makes more sense now#2020-05-0322:09mruzekwAnd I'd probably invoke that in my http handler#2020-05-0322:09eoliphantits a wildly more complex example of something like
(def add2 (partial + 2))
(add2 2) ; => 4
#2020-05-0322:10eoliphantexactly#2020-05-0322:10mruzekwGotcha, so p/parser is the higher order fn#2020-05-0322:11eoliphantforgot about the env-wrap-plugin that @danvingo mentioned. Same principle. Easier way to dynamically muck with the env#2020-05-0322:11eoliphantyep#2020-05-0322:12mruzekwGotcha, thanks#2020-05-0322:13mruzekwAnd then for grabbing data after a db mutation?#2020-05-0322:22eoliphantthat particular case is a little tricker. I’ll have to find the code, but you can basically write your own plugin and say ā€˜refresh’ the db value after a mutation takes place. Mine was pretty straightforward, but you then get into stuff like ā€˜well was that one that i cared about’ etc. There may be ways (pathom packs a ton of metadata into the env), to further optimize it, like say determining if what i’m about to resolve, is linked to a mutation that happened. It can get a little weird because you can end up with other issues, in my more simplistic attempt you could end up calling a resolver with a new db value even though what was being resolved had nothing to do with the change that was made, and then you lose the ā€˜stable db’ view even when it’s not necessary
#2020-05-0322:23mruzekwI see. So would you recommend passing the connection and the latest db value (when the parser is first called) to the parser env?#2020-05-0322:23mruzekw(Not assuming use of any plugins)#2020-05-0322:24eoliphantyeah I’d start with that for sure, then see how your use cases play out#2020-05-0322:24mruzekwOkay, cool. Thank you šŸ™‚ I was a bit lost on how the env was actually constructed and where#2020-05-0322:24eoliphantsimplicity first šŸ™‚ lol . np at all#2020-05-0322:25mruzekwAgreed#2020-05-0322:37mruzekwAre there any example projects on GH with Pathom usage? I know it's typically used in Fulcro#2020-05-0322:38eoliphantyeah the default fulcro setup is probably the best way to get started as it shows how to wire in requests from the client, etc.#2020-05-0322:41mruzekwOkay, I'll see if I can find that#2020-05-0322:41eoliphanthere it is https://github.com/fulcrologic/fulcro-template#2020-05-0322:41mruzekwThanks#2020-05-0322:42eoliphantnp and the pathom setup is there: https://github.com/fulcrologic/fulcro-template/blob/master/src/main/app/server_components/pathom.clj#2020-05-0403:59Matthew@wilkerlucio I was using the async-parser, then I read the docs further and saw that the parallel-parser is the recommended approach. The result from the parallel parser gave me the single core async channel that I expected. I was a little confused reading the docs, but I think it’s just that they haven’t been consistently updated as of yet.#2020-05-0404:00wilkerlucio@matthew_lefebvre I suggest you keep on the async-parser unless you have a lot of paralellizable things (which is not common)#2020-05-0404:01wilkerluciosorry the confusing docs, they are a result of trying to upgrade it step by step#2020-05-0404:02MatthewIs your recommendation performance based, base on a node (single threaded) environment?#2020-05-0404:03wilkerlucioyeah, the parallel parser is much more complex, so debugging around it is much harder#2020-05-0404:03MatthewI’m pretty sure nothing parallel is happening in the JS runtime unless there are workers involved in the implementation #2020-05-0404:04wilkerlucioand if you don't have concurrent situations (can be on a single thread), it is slower#2020-05-0404:05wilkerlucioit cna go in parallel on node, when you do async things, for example, you can have multiple async requests (external http requests for example) going in parallel, even on the single thread#2020-05-0404:23MatthewYea sorry, I understand what your saying if you mean all the IO threads running outside of the event loop. It’s interesting that you suggest the async parser. I didn’t really understand the way my parse results were returned to me, then the parallel parser just sort of did exactly what I was expecting.#2020-05-0404:28MatthewWith the async parser the channel is inside a returned map. Where as with the parallel parser the entire returned value is a channel that you take from and all your data is there exactly in the shape your expecting. #2020-05-0404:28MatthewI don’t think my async parser was configured correctly #2020-05-0405:30wilkerlucio@matthew_lefebvre with the async parser you must async-reader2#2020-05-0405:38sergey.shvets@wilkerlucio is there a special reader needed to be included to enable eql params support?#2020-05-0405:51sergey.shvetsI'm unable to get params in :ast for ident. https://github.com/edn-query-language/eql#parameters. I tried both syntaxes.#2020-05-0406:04wilkerlucio@bear-z they are supported by default, the main issue people have with this is the expectation about where this should appear, can you send me the query you are trying to process?#2020-05-0406:21sergey.shvets
(def sub-list-query ['({[:hand-and-foot.game-rounds/where {:hand-and-foot.game-rounds/number 1}] [{:hand-and-foot.game-rounds/list [:hand-and-foot.game-rounds/id]}]}
                         {:hand-and-foot.games/id "TKSbTbE6O5Q0U89JhuHc"})])
#2020-05-0406:48sergey.shvetsSorry, one more question: can you nest joins more than one level?#2020-05-0406:59wilkerlucio@bear-z yes you can, and your query, the param will come ont he ident join, but you are probably trying to read them from the child level#2020-05-0406:59wilkerlucioothen than that the query seems write, altough there is a different way that also works that I find cleaner:
(def sub-list-query
  [{'([:hand-and-foot.game-rounds/where {:hand-and-foot.game-rounds/number 1}] {:hand-and-foot.games/id "TKSbTbE6O5Q0U89JhuHc"}) 
    [{:hand-and-foot.game-rounds/list [:hand-and-foot.game-rounds/id]}]}])
#2020-05-0407:00wilkerluciothis is cleaner IMO because it puts the param on the side of what they target#2020-05-0407:00sergey.shvetsThanks, how do I access them in resolver if they're added on ident level?#2020-05-0407:00wilkerluciowhat people usually do is to write a plugin to propagate params down via env#2020-05-0407:01wilkerlucioon the #fulcro channel that question has appeared multiple times, probably someone there has the snippet for it :)#2020-05-0407:01sergey.shvetsCool. I'll research that path then.#2020-05-0407:04sergey.shvetsBack to joins: I'm trying to do a query that will read games->rounds->moves . I have resolver for rounds that takes as input game-id. This works fine. The second resolver for moves requires both game-id and round-id and it unable to find it. Both rounds and moves expect to return list of items.#2020-05-0407:05sergey.shvetsWill the response from level 0 be available on level 1 or I have to somehow propagate it?#2020-05-0407:07wilkerlucionot automatically, but thats a pattern I have done a couple times, on the rounds resolver you can pass the game-id down to the next entity#2020-05-0407:07wilkerluciothat's a reason why the big names are so important#2020-05-0407:08sergey.shvetsSo, I just need to include it in return of the rounds and then it will figure it out?#2020-05-0407:09wilkerlucioyup, it just needs the data to exist on the entity, or have a path to find it#2020-05-0407:09sergey.shvetsI tried to pass it as :pathom/context but that doesn't seem to work#2020-05-0407:09wilkerlucionot context, you really just mesh it down#2020-05-0407:09wilkerlucioexample:#2020-05-0407:09sergey.shvetsI think I get it now.#2020-05-0407:10sergey.shvetsBtw, is :pathom/context a correct keyword? All the rest of keywords are prefixed with core or connect namespace.#2020-05-0407:10wilkerluciosomething like:#2020-05-0407:10wilkerlucio
(pc/defresolver rounds [env {:keys [game/id]}]
  {::pc/input  #{:game/id}
   ::pc/output [{:game/rounds [:game/id
                               :round/whatever]}]}
  {:game/rounds (->> (get-rounds rounds id)
                     (mapv #(assoc % :game/id id)))})
#2020-05-0407:12sergey.shvetsGot it. Will try it and report the results. Thank you so much for the help @wilkerlucio. I would probably spend weeks to figure this out on my own!#2020-05-0407:14wilkerluciono worries, glad to help#2020-05-0505:44sergey.shvets@wilkerlucio Thanks for your suggestions yesterday. Both worked like a charm.#2020-05-0505:45sergey.shvetsHere is my pass-down params script for reference:
(def params-down-pathom-plugin
  {::p/wrap-read (fn [reader]
                   (fn [env]
                     (if-let [params (get-in env [:ast :params])]
                       (reader (as-> 
                                (map (fn [child]
                                       [(:dispatch-key child) params])
                                     (get-in env [:ast :children])) $
                                 (into {} $)
                                 (update env :parent-params merge $)))
                       (reader env))))})

(defn parse-query-params
  "Parses query and ast to see if there any params related to the current query. Returns map with params or empty map."
  [env]
  (let [dispatch-key (get-in env [:ast :dispatch-key])
        params       (get-in env [:parent-params dispatch-key] {})]
    params))
#2020-05-0513:34ouvasamHi do you have an example of query to use with it ? Thanks#2020-05-0515:04sergey.shvets
[{([:hand-and-foot.games/id "TKSbTbE6O5Q0U89JhuHc"] {:with "params"})
  [:hand-and-foot.games/name
   :hand-and-foot.games/players
   :hand-and-foot.games/status 
   ({:hand-and-foot.games/rounds 
      [:hand-and-foot.game-rounds/id
       :hand-and-foot.game-rounds/number]} {:limit 2 :order-by :hand-and-foot.game-rounds/number})]}]
#2020-05-0515:06sergey.shvetsInside a resolver you need to call parse-query-params and it will figure out if there are any params that are related to this resolver. It rolls down params only one level to the immediate children of a node where params have been applied.#2020-05-0515:07sergey.shvetsIt shouldn't be too hard to roll it down all the way, but I think it will do more harm than good.#2020-05-0518:38wilkerlucioanother option is roll it down via :env instead of changing the AST, this way you can have "accumulated" (could go deep merging all the way down) and the current#2020-05-0518:43sergey.shvetsI haven't changed ast, I just added a new parameter to env#2020-05-0521:02wilkerluciooh, right, I misread it šŸ‘#2020-05-0612:24ouvasamMany thanks !#2020-05-0615:21sergey.shvetsyou welcome!#2020-05-0505:46sergey.shvetsUnless you have same dispatch keys with different params in your query it will support multiple params in different places of a query.#2020-05-0515:10sergey.shvetsCan query have multiple idents for one join? Or you need to always create some "join node" that will accept multiple values as ident param and set them to the env?#2020-05-0515:11sergey.shvetsI had a thought that probably the requirement for Idents can be loosened a bit from "only 2 arguments" to even number of arguments and I feel like this will open more opportunities, without changing parser/ast too much#2020-05-0515:19kszaboI proposed this before, but was rejected: https://github.com/edn-query-language/eql/pull/9#2020-05-0515:20kszaboI have since wrote support for it in Pathom, so it works with an extra reader#2020-05-0515:21kszabobut it brings up a lot of questions on the client side, especially data normalization#2020-05-0515:22kszabobut with this syntax you can write quries like:
[{{:latitude  42
   :longitude 42.3}
 [:location/name]}]
#2020-05-0515:22kszabowithout having to think up of an ident beforehand#2020-05-0515:23kszaboIn my mind this is still way cleaner than the pathom parser specific :pathom/context hack#2020-05-0515:29kszabobut I understand the concerns, as this diverges greatly from the status quo.#2020-05-0516:28sergey.shvetsThanks for sharing. I haven't seen those before, since I'm very new to pathom and eql in general. I was planning to implement something similar to what Wilker suggesting here https://github.com/wilkerlucio/pathom/issues/140 with expand-thing-compound and that should cover my problem. But on the other hand, allowing longer vectors in idents doesn't seem like a huge change to me. Of course, there might be nuances that I don't grasp yet. Just wanted to share an idea. The workaround with expand-thing-compound is very simple one and pretty much need one extra resolver per project.#2020-05-0517:01kszaboone extra resolver per compound ident, I think#2020-05-0517:01kszaboin my case I have 26 of these, of varying combinations šŸ™‚#2020-05-0517:02kszaboso this blows up quickly#2020-05-0517:03sergey.shvetsOh, I get what you're saying. We can't specify outputs for resolver without knowing what will be inside the ident param#2020-05-0517:16sergey.shvetsso, the best way for now is to use this:
[{([:init-context/a 42] {:pathom/context {:init-context/b "foo"}}) [:want/this]}]
#2020-05-0517:17sergey.shvetsI like your suggestion,
[{{:init-context/a 42 :init-context/b "foo"} [:want/this]}]
but to minimize changes to spec I think it would be better to keep it as this
[{[:init-context/a 42 :init-context/b "foo"] [:want/this]}]
#2020-05-0517:17sergey.shvetsIf I'm not missing something then you just need an extra reader and loosen the spec if it has a validation for exactly two to the even?#2020-05-0517:55sergey.shvetsHow did you write reader for map ident in Pathom? Have you changed eql specs or pathom doesn't check for validity of the query beforehand?#2020-05-0518:01kszaboyup, the whole ordeal#2020-05-0518:02kszaboI dislike the a 2+ vector idea as this join context as I call it is an associative ideas which we already have a perfectly fine representation for#2020-05-0518:02kszaboif I diverge from EQL, might as well use the best datastructure for it#2020-05-0518:03kszaboof course the idea of how to determine the dispatch key came up, which is just currently ffirst, so it depends on the hashmap in-memory ordering guarantees, but this hasn’t bit me yet#2020-05-0518:07sergey.shvetsIsn't dispatch key for ident the whole vector? (or map in your case)#2020-05-0518:09sergey.shvetsActually, disregard. I see that it isn't. It is :first#2020-05-0518:13sergey.shvetsI guess for now, I'll have to bite the bullet and use :pathom/context šŸ™‚#2020-05-0518:18kszaboyeah, the coming foreign parser support also (ab)uses that#2020-05-0518:18sergey.shvetswhat is foreign parser?#2020-05-0518:25kszabo🤫 šŸ™‚#2020-05-0518:26kszabohttps://github.com/wilkerlucio/pathom/blob/query-planner/src/com/wsscode/pathom/connect/foreign.cljc#L103#2020-05-0518:29sergey.shvetsActually, it seems like pathom context doesn't work for my case or I'm doing it totally wrong.#2020-05-0518:31sergey.shvetsI have a resolver that has input #{:parent/id child/id} and trying to call it: [{'([:child/id "a2coh2mnsIyd35CL8PVv"] {:pathom/context {:parent/id "TKSbTbE6O5Q0U89JhuHc"}}) [:child/data]}]#2020-05-0518:31sergey.shvetsand it isn't finding the proper resolver#2020-05-0518:31sergey.shvetsDamn.#2020-05-0516:50souenzzoWhen I connect my app into pathom-viz, autocomplete works but index explorer don't How can I debug it? public API URL: public code: https://github.com/souenzzo/caderninho/blob/master/src/br/com/souenzzo/caderninho.clj#L24#2020-05-0517:34souenzzoSolution it's something with transit/unknown tags (sp/setval (sp/walker fn?) sp/NONE response)#2020-05-0517:34souenzzoFixed by removing any function from response#2020-05-0518:56sergey.shvetsI have a resolver that has input #{:parent/id :child/id} and trying to call it: [{'([:child/id "a2coh2mnsIyd35CL8PVv"] {:pathom/context {:parent/id "TKSbTbE6O5Q0U89JhuHc"}}) [:child/data]}] Get not-found in response. Is this supposed to work? Because I can't see what am I doing wrong.#2020-05-0521:15wilkerlucio@bear-z this should work, can you also try adding '* to your query? so you can see all data available#2020-05-0521:15wilkerlucio[{'([:child/id "a2coh2mnsIyd35CL8PVv"] {:pathom/context {:parent/id "TKSbTbE6O5Q0U89JhuHc"}}) [:child/data '*]}]#2020-05-0521:21sergey.shvetsNo luck: Here is my query: https://take.ms/05ecK And here is the resolver it supposed to call: https://take.ms/6IOp1#2020-05-0521:22wilkerlucioare you using the open-ident-reader?#2020-05-0521:22sergey.shvetsident-reader :
(p/async-parser
                               {::p/env     {::p/reader               [p/map-reader
                                                                       pc/async-reader2
                                                                       pc/ident-reader
                                                                       p/env-placeholder-reader]
                                             ::p/placeholder-prefixes #{">"}}
                                ::p/mutate  pc/mutate-async
                                ::p/plugins [(pc/connect-plugin {::pc/register pathom-resolvers})
                                             p/error-handler-plugin
                                             p/trace-plugin
                                             params-down-pathom-plugin]}))
#2020-05-0521:25sergey.shvetsopen-ident-reader does solve it.#2020-05-0521:25sergey.shvetsThank you!#2020-05-0612:30ouvasamI have a problem using crux and pathom and i can't find the problem. I have a resolver that call a function with a datalog query in crux. The first time it is executed all his fine, the crux function is called and return the results. If i call it within a few seconds, the resolver is not called anymore or at least no error is seen. If i call the crux function from the ring query directly it works perfectly. So it seems that crux is ok. If i remove the crux call in the resolver it is call normally and return correctly I have no idea what could be wrong here ? If someone has an idea ? Many thanks#2020-05-0612:49ouvasamHere is the code for the parser
(def parser
  (p/async-parser
   {::p/env     {::p/reader               [p/map-reader
                                           pc/async-reader2
                                           pc/ident-reader
                                           p/env-placeholder-reader]
                 ::p/placeholder-prefixes #{">"}}
    ::p/mutate  pc/mutate-async
    ::p/plugins [(pc/connect-plugin {::pc/register app-registry})
                 p/error-handler-plugin
                 p/trace-plugin
                 params-down-pathom-plugin]}))
#2020-05-0612:49ouvasamThe code for the ring handler
(defn query
  [query]
  (let [params (clojure.walk/keywordize-keys (:params query))
        q (:query params)]
    (println "ring query" q)
    ; (println (crux-api/get-concept-translations-by-jurisdiction 1 (rand-nth (range 730 750))))
    {:status 200
     :body
     (encode-transit-json
      (<!! (parser params (clojure.edn/read-string q))))}))
#2020-05-0612:51ouvasamThe code for the resolver that only work "sometime"
(pc/defresolver by-jurisdiction-id [{:keys [] :as env} params]
                {::pc/input #{:translation/query}
                 ::pc/output [:translations
                              [:translation/id
                               :translation/language
                               :translation/value
                               :translation/status
                               :jurisidction/id
                               :concept/id
                               :app/type]]}
                (let [query (parse-query-params env)
                      jurisdiction-id (:jurisdiction/id query)
                      concept-id (:concept/id query)
                      results (crux-api/get-concept-translations-by-jurisdiction)]
                    {:translations []}))
#2020-05-0612:51ouvasamthe same without the crux call work perfectly#2020-05-0613:55ouvasamI also ask on crux channel but can't find a solution ...#2020-05-0614:41ouvasamanybody has an idea ?#2020-05-0615:17Chris O’DonnellHave you tried wrapping the resolver body in a try-catch to see if an exception is getting silently swallowed?#2020-05-0616:02dvingoit looks like your resolver's output is invalid for a join#2020-05-0616:02dvingo{:translations [..]}#2020-05-0616:34ouvasamyes i tried to catch error but nothing#2020-05-0616:35ouvasamoutput is still the same each time#2020-05-0616:35ouvasami did try with something that return the same#2020-05-0616:45dvingoyour output should be:
[{:translations
                              [:translation/id
                               :translation/language
                               :translation/value
                               :translation/status
                               :jurisidction/id
                               :concept/id
                               :app/type]}]
#2020-05-0708:02ouvasamMany thanks for the answers, but even with this i still have troubles #2020-05-0708:02ouvasam:(#2020-05-0712:23ouvasami found my problem ! I had two resolver with the same output signature but one with an input. It did a conflict apparently. Many thanks for your support#2020-05-0613:12Eugen#2020-05-0613:12Eugenreplace Fulco with pathom#2020-05-0613:18cjmurphy@eugen.stan I know I suggested you come here, but now I'm not so sure what the API you are interested in would be. With Pathom there is no static API as such because Pathom receives EQL and sends back the query results.#2020-05-0613:18EugenI'm asking about HTTP API - the one the external developers would use#2020-05-0613:18Eugenso Fulcro it is#2020-05-0613:20cjmurphyWell Fulcro sends across the EQL, that it created from the composition of Fulcro component's queries. So again I'm not sure where the static API would be.#2020-05-0613:36kszabothe recently released pathom-viz electron/stanadlone app is close as you can get currently without writing any code yourself#2020-05-0613:37kszabohttps://github.com/wilkerlucio/pathom-viz/releases#2020-05-0613:37kszabo@eugen.stan fyi ^#2020-05-0613:39EugenThanks, something like that . it would be great if it can generate docs to be served by static web servers#2020-05-0613:42kszaboyeah, there is no static site yet, but you can use the server-side rendering cljs of the above code to implement it#2020-05-0613:43Eugenthanks, I will keep that in mind. In the mean time I filed a question issue . It should remain there as docs at least.#2020-05-0614:41ouvasamanybody has an idea ?#2020-05-0616:59Michael WHow do you tie 2 different datasources to each other? I have 2 different apis, one works off id (ddi), I have that fully wired with resolvers, and can query it. The other api (aci) works off ip address, I have that fully wired with resolvers and can query it. How do I tell the first api that the ip field is what it needs to query from the other api and not the id field?#2020-05-0617:00Michael W
(parser {} [{[:ip/ip "10.4.94.48"] [:ip/ip :ip/pool :ip/subnet :ip/aci]}])
{[:ip/ip "10.4.94.48"] #:ip{:ip "10.4.94.48", :pool "17", :subnet "59", :aci :com.wsscode.pathom.core/not-found}}
#2020-05-0617:00Michael W
(parser {} [{[:aci/ip "10.4.94.48"] [:aci/ip :aci/mac]}])
{[:aci/ip "10.4.94.48"] #:aci{:ip "10.4.94.48", :mac "00:50:56:BB:ED:01"}}
#2020-05-0617:11souenzzoadd (pc/alias-resolver2 :ip/ip :aci/ip) to your registry @michael819#2020-05-0617:14Michael Wis that a plugin?#2020-05-0617:24Chris O’DonnellNo, it's a resolver. It is equivalent to the following two resolvers:
(defresolver ip->aci-resolver [_ input]
  {::pc/input #{:ip/ip}
   ::pc/output [:aci/ip]}
  {:aci/ip (get input :ip/ip)})

(defresolver aci->ip-resolver [_ input]
  {::pc/input #{:aci/ip}
   ::pc/output [:ip/ip]}
  {:ip/ip (get input :aci/ip)})
#2020-05-0617:34Michael WI tried that and it didn't work#2020-05-0617:35sergey.shvets@michael819 I'm not sure I fully understand your question, but it seems like this might help: https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/plugins.html#_example_shard_switch#2020-05-0617:37Michael WThanks, let me play with it some more, I know I am really close since I can query both apis independently I just need to figure out how to wire them together.#2020-05-0617:37Michael WI'm looking at the shard thing now @bear-z#2020-05-0617:38sergey.shvetsAre your apis independent or there is some field that you can link them on?#2020-05-0617:39sergey.shvetsBecause what souenzzo suggests should do the trick if you just need to connect one field to the other.#2020-05-0617:39Michael WYeah they have a common field, the ip address#2020-05-0617:40sergey.shvetsWhat is the end query you're trying to do?#2020-05-0617:41sergey.shvets
(parser {} [{[:ip/ip "10.4.94.48"] [:ip/ip :ip/pool :ip/subnet :ip/aci]}]) 
I think you need to change it to
(parser {} [{[:ip/ip "10.4.94.48"] [:ip/ip :ip/pool :ip/subnet :aci/ip :aci/mac]}])
And use alias-resolver2. That should work
#2020-05-0617:42Michael WSo I have a DDI (DHCP/DNS/IPAM) that has bad mac addresses in it. I am trying to query for mac addresses that don't match between the systems, they both share the ip as a key.#2020-05-0617:43Michael WI get nils for :aci/ip and :aci/mac#2020-05-0617:44sergey.shvetshave you registered your (pc/alias-resolver2 :ip/ip :aci) to the parser#2020-05-0617:44Michael WYes#2020-05-0617:45Michael W
(def ip-aci-alias (pc/alias-resolver2 :ip/ip :aci/ip))$
$
(def resolvers [aci-ip aci-mac$
                subnet-id subnet-ip subnet-start subnet-end$
                pool-id pool-ip$
                ip-id ip-ip ip-aci-alias])$
#2020-05-0617:45Michael WI have $ for end of line#2020-05-0617:45sergey.shvetsCan you confirm that resolver with #{:aci/ip} input gets called?#2020-05-0617:46sergey.shvetsThere is a pathom-viz standalone app that can make your debugging much easier#2020-05-0617:46sergey.shvetshttps://github.com/wilkerlucio/pathom-viz#2020-05-0617:55Michael WAh hah! Got it working, thanks @souenzzo @bear-z#2020-05-0617:56Michael W
(parser {} [{[:ip/ip "10.4.94.48"] [:ip/mac :aci/mac]}])
{[:ip/ip "10.4.94.48"] {:ip/mac "05:42:39:84:62:62", :aci/mac "00:50:56:BB:ED:01"}}
#2020-05-0617:59sergey.shvetsNice! What was the problem?#2020-05-0618:00Michael WI had a typo in some other resolver where it was :aci/id not :aci/ip#2020-05-0618:01sergey.shvetsGlad you found it!#2020-05-0618:01Michael WThe alias ultimately glued the 2 systems together, so thanks again for the help, made my day!#2020-05-0704:57sergey.shvets@wilkerlucio Is there a special parser/reader required so ::pc/transform (pc/transform-auto-batch 10) works? I'm running on cljs and have async-parser, but all the requests are running consequently: https://take.ms/kNLoL#2020-05-0705:02wilkerluciois this a recursive query? the batch only works for sibling entities, when driling down it currently just chains the process#2020-05-0705:06sergey.shvets
[{([:hand-and-foot.games/id "TKSbTbE6O5Q0U89JhuHc"] {:with "params"})
  [:hand-and-foot.games/name
   :hand-and-foot.games/players
   :hand-and-foot.games/status 
   {:hand-and-foot.games/rounds 
    [:hand-and-foot.game-rounds/id
     :hand-and-foot.game-rounds/number
     
     {:hand-and-foot.rounds/moves 
      [:hand-and-foot.moves/number 
       :hand-and-foot.moves/player]}]}]}]
#2020-05-0705:06sergey.shvetsThese are sibling entities and they are independent#2020-05-0705:52sergey.shvetsAm I right that you can only have one ident per query?#2020-05-0705:52sergey.shvetsIf you have multiple inputs then you need to use :pathom/context ?#2020-05-0706:37sergey.shvetsAha, I just now realized that I can pass params to the (parser) through env. That makes my life muuuuch easier and queries are more readable now šŸ™‚#2020-05-0715:35uwoJust out of curiosity, is there a small library anywhere for turning datomic pull expressions into ASTs -- if not, is there a library that deals only with transforming EQL to an AST ( though I realize datomic pull isn't a proper subset of EQL).#2020-05-0715:37uwookay, nvm. Found it for EQL https://github.com/edn-query-language/eql/blob/master/src/edn_query_language/core.cljc#2020-05-0716:04dvingohttps://github.com/tonsky/datascript/blob/master/src/datascript/pull_parser.cljc#2020-05-0717:20souenzzohttps://github.com/souenzzo/eql-datomic/#2020-05-0720:24uwoy'all rock. thanks#2020-05-0719:52BrianHow can I pass in a vector of things into a resolver? For instance, I have a bunch of item id's. I would like to send 2+ ids into my resolver and have it determine the similarity between those ids (meaning I can't send in 1 idea at a time). Hopefully this example shows what I am trying to do:
(defresolver similar-traits [env {:keys [[item/id]]}]
  {::pc/input  #{[:item/id]}
   ::pc/output [:result]}
  {:result (get-result <ids somehow>)})
#2020-05-0719:53sergey.shvetsThe second parameter in ident can be anything. So you can have ::pc/input #{:item/ids} and then use ident like [[:item/ids [:id1 :id2]]]#2020-05-0719:58BrianCan you please write out the whole resolver @bear-z? I can't seem to get it working. I'd like the resolver to know that it's a vector of :item/ids not just an object with an :item/ids key#2020-05-0720:04sergey.shvetsI don't think you can do that unless it is a batch resolver. https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/resolvers.html#_n1_queries_and_batch_resolvers. Try this:
(defresolver similar-traits [env input]
  {::pc/input  #{:item/id}
   ::pc/output [:result]
   ::pc/batch? true}
  {:result (get-result <your input will have a sequence of :item/id>)})
But batch resolvers will only be called when you have a sibling query (meaning they appear in joins), I don't think you can run them from the root.
#2020-05-0720:05sergey.shvetsFor the root, you can just create this extra key that will expect to receive a vector of :item/id#2020-05-0720:37Brian@bear-z okay I think I understand now. One thing the docs don't provide is an example of how to actually pass that data into the parser. Do you know how to do that? I tried something like
(<!! ((du/get-parser) {} [{[{:item/id 1}
                            {:item/id 2}
                            {:item/id 3}]
                           [:result]}]))
but this yielded an error
#2020-05-0720:43sergey.shvetsThat won't work because your query is invalid. Ident always a tuple of 2 elements. Is there a reason that you want resolver to get :item/id separately and not as a list? Because I think you're overcomplicating it. If it always takes a list and returns some computed result, why would you add an overhead of batching and stuff?#2020-05-0720:55BrianI don't know that batching is the answer I've only just learned about it. I just want to be able to send a vector of n {:item/id <id>}s into the parser and have a result computed. And I'm stuck on the n part. I have [{:item/id 1}{:item/id 2} ... {:item/id n}] . I can do something like
(defresolver ixn-similar-traits [env [:keys [item/ids]]]
  {::pc/input  #{:item/ids}
   ::pc/output [:result]}
  {:result (get-result ids)})
and then
(<!! ((du/get-parser) {} [{[:item/ids
                              [{:item/id 1}
                               {:item/id 2}
                               {:item/id 3}]]
                             [:result]}]))
but the resolver won't be able to check to ensure that :item/ids is actually a vector of maps with :item/id keys. I would like to further describe to the resolver that it should only work when receiving a vector of maps with :item/id keys. Expressed in edn [{:item/ids [:item/id]}]
#2020-05-0721:19sergey.shvetsYou can add a spec inside resolver or even a simple assert#2020-05-0721:23sergey.shvetssomething like that:
(s/def ::item-map (s/keys :req [:item/id]))
(s/def ::items-coll (s/coll-of ::item-map))
And then inside your resolver simple: (s/valid? ::items-coll input) Full guide is here: https://clojure.org/guides/spec#_collections
#2020-05-0814:33BrianThank you @bear-z!#2020-05-0909:54TuomasA newbie here. I’m developing a fulcro app, with http-kit server+pathom backend locally. I’m deploying to datomic cloud solo as and exposing the api as an ion. The frontend is in a s3 website bucket. The idea is to get a static website, authorize with tokens and work with a pathom api that runs on a datomic cloud system and is exposed as an ion. Everything is fine locally, but after deploy I’m getting:
["^ ","~:session/current-user","~:com.wsscode.pathom.core/reader-error","~:com.wsscode.pathom.core/errors",["~#cmap",[["^0"],"class java.lang.NullPointerException"]]]
I made a fast PoC few months back that worked fine. Any tips for debugging? I’m suspecting this has something to do with initializing the db but thought I’d ask here first if it’s something obvious.
#2020-05-0916:14TuomasI got it. Pretty obvious. So if you are using a
p/error-handler-plugin
then your resolvers are wrapped in try catch and you get the errors back in that form. Similar to GraphQL. Also, not all clojure functions can handle nil, for example string/split and <. Newbies should always test forms with nil to avoid NullPointerExceptions
#2020-05-0922:18wilkerlucioone other tip if you need to debug in prod, set ::p/process-error (fn [env err] (log-somehow err) (p/error-message err)) on your env to have more visibility on the errors#2020-05-1114:39Aleedwhy are mutations referenced by a symbol rather than a keyword? i.e.Ā `'user/create`Ā rather thanĀ `:user/create`#2020-05-1114:41Aleedalso, how to people handle errors client-side? from apollo graphql I’m used to having separate error property, i.e. the query would return {data, errors, loading}, but in pathom error is returned in place of the result. not immediately obvious to me why that is and how to best handle that#2020-05-1114:59wilkerluciohello @alidcastano, mutations are symbols to separated than from reads, since mutations and resolvers do fundamental different operations (read vs write), this also aligns with Clojure itself, where keywords are usually a way to read information, while lists with symbols are calls. about the error, pathom by default will give the error in a separated property on the map, similar to GraphQL, but IMO for UI development its easier to have the errors close to the entity that failed, there is a helper to do that p/raise-errors#2020-05-1116:12AleedI was seeing the error as part of mutation result (i.e. {user/create *some error*} ), but perhaps that’s because I have the p/error-handler-plugin configured i tried p/raise-errors but didn’t see it do anything. is it meant to wrap the parser output?#2020-05-1116:12Aleedi.e. (p/raise errors (parse env eql))#2020-05-1116:14Aleedre: symbols. I figured that was the case. just makes it a bit more awkward to use in frontend, since you have to quote/unquote . likely just have to get used to that#2020-05-1120:34wilkerluciooh, mutation errors always go on the mutation itself, resolver errors will use the gql style#2020-05-1120:59AleedI’m trying to figure out best way of thinking about this. for comparison, in apollo graphql if a mutation errors out you’d catch the error and data is only returned when promise is resolved. but in pathom mutations result can resolve to both data or errors. so should I always check that result is not an error before trying to use the actual returned data? (I guess that’s the obvious thing to do, but wondering if there’s any non-obvious approach Im not considering)#2020-05-1213:02souenzzo[2.3.0-alpha5] reader2 and reader3 do different things in ast
(for [reader [pc/reader2
              pc/reader3]]
  (let [parser (ps/connect-serial-parser
                 {::ps/connect-reader reader}
                 [(pc/resolver `b
                               {::pc/output [:b]}
                               (fn [env input]
                                 {:b (assoc env :keys (keys env))}))])]
    (-> (parser {} [{:>/a [{:b [{:ast [:dispatch-key]}]}]}])
        :>/a 
        :b 
        :ast)))

=> ({:dispatch-key :b} {:dispatch-key :>/a})
#2020-05-1217:15dvingodoes pathom not let you return a set for a to-many join?#2020-05-1217:15dvingoi'm seeing a join work when returning a vector, but not for a set#2020-05-1218:18wilkerlucioto many joins are always vector#2020-05-1218:18wilkerlucio@souenzzo reader3 process is very different from the reader2, and some things will have to work differently#2020-05-1218:22dvingocool, thanks#2020-05-1218:23dvingoahhh i just learned that the env contains the parser šŸ˜„ as a relative newcomer to pathom that wasn't obvious.. makes some use cases a lot simpler (or even possible) to implement#2020-05-1219:41dvingoThis is a bit wordy, but I'm not sure how else to explain it. I have a design where I want to generate some data for a user on the first request for that data, but if they update any of the data I store only the updates (the stored data will be sparse compared to the entire generated data set) A user has a collection of "template" documents. On each request I generate an entity (and it's tree of child properties) in the response for each of the template documents. If the user updates one of the generated entities, then I persist only that entity (with its children). On subsequent requests I will return any existing entities but generate any remaining entities that are in the template list. These are the two main resolvers:
(pc/defresolver user-habit-records-resolver [_ _]
  {::pc/output [{:user/habit-records
                 [{:habit-record/id [:habit-record/id]}]}]})

(pc/defresolver habit-record-resolver [_ _]
  {::pc/input  #{:habit-record/id}
   ::pc/output [:server/message :server/error?
               :habit-record/id
               :habit-record/date
               :habit-record/state
               {:habit-record/habit-id [:habit/id]}
               {:habit-record/task-records [:task-record/id]}]})
user-habit-records-resolver returns a vector of a mix of just the habit-record/id and a tree of data that matches the query for habit-record-resolver for missing entities. This is currently working, pathom is correctly resolving all the refs in the generated data and for the id case. My question is: is this supported? Or is this working by "accident" because of the current implementation of pathom. I don't want to get caught in the future if the implementation should change and break this setup.
#2020-05-1219:55Eric IhliWorking through the Pathom doc examples and I'm stuck on `defmutation`. I define a mutation as in the docs. I see it's a macro that does ~ `(def ~sym ,,, (fn ~sym ~arg /cdn-cgi/l/email-protection)`, so I expect it to create a function that returns the body I pass to `defmutation`. But when I run the function, it returns `nil` and I can't identify why. <https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/connect-mutations.html#_creating_mutations> ```(pc/defmutation send-message [env {:keys [message/text]}] {::pc/sym 'send-message ::pc/params [:message/text] ::pc/output [:messag/id :message/text]} {:message/id 123 :message/text text}) (def my-app-registry [send-message]) (def parser (p/parallel-parser {::p/env {::p/reader [p/map-reader pc/parallel-reader pc/open-ident-reader p/env-placeholder-reader] ::p/placeholder-prefixes #{"&gt;"}} ::p/mutate pc/mutate-async ::p/plugins [(pc/connect-plugin {::pc/register my-app-registry}) p/error-handler-plugin p/request-cache-plugin p/trace-plugin]})) ;; Blows up with "Invalid expression " (&lt;!! (parser {} [(send-message {:message/text "Hello Clojurst!"})])) ;; Returns nil (send-message {:message/text "Hello Clojurst!"})``` Anyone have an idea why? Where am I going astray?#2020-05-1219:57dvingoprobably just need to quote the symbol:
(<!! (parser {} [(list 'send-message {:message/text "Hello Clojurst!"})]))
#2020-05-1220:00Eric IhliThat's it. Huh. I swear I tried quoting the entire expression (<!! (parser {} '[(send-message {:message/text "Hello Clojurst!"})])) and it failed. But it's working now.#2020-05-1220:03Eric IhliHm. Well heck. Wish I could go back in time and figure out what was really going on earlier when the quote wasn't working. Oh well. Thanks!#2020-05-1220:05dvingonp, it happens!#2020-05-1317:28Eric IhliDoes something special need to be done to reload a resolver after I edit it while working in the REPL? I made a change to the return value of a resolver and then re-evaluated the buffer where my resolver and parser are defined. But running a query afterwards doesn't pick up the change.#2020-05-1405:57fjolneif you’re calling the pathom parser directly then just redefining it should work fine, defresolver expands to def but for more complex apps you might want to go with some state management library (mount, integrant, etc) + with tools.namespace for code reload, because often there’re things which have lifecycle semantics (like a server) and if your pathom parser is accessed via the server, you need to reload the server as well #2020-05-1318:35folconYes, you have to reload your parser, your parser caches your resolver definitions, so you need to update it, just make sure you’re doing that =)… You might be calling an older version of the parser, I’ve done that before…#2020-05-1319:39Ī»ustin f(n)What is the best way to add a GraphQL api support to an existing Pathom setup? As in, supports GraphQL requests, not just consuming an existing GraphQL data source.#2020-05-1319:41Ī»ustin f(n)I have a legacy Javascript/React ui I need to support that I want to introduce a better data management and request layer to. To keep things more simple for the UI devs, I was hoping to use some sort of GraphQL client library like https://www.apollographql.com/docs/react/#2020-05-1319:42Ī»ustin f(n)Is https://github.com/denisidoro/graffiti Basically the only option? I was hoping not to add in another layer of library that might not be updated. Plus, wouldn't it include re-writing whatever pathom resolvers already exist?#2020-05-1321:21wilkerluciograffiti is the only option I know for that currently#2020-05-1415:50Eric Ihli> https://wilkerlucio.github.io/pathom/#_the_environment_plugin > Typically the parsing environment will need to include things you create and inject every time you parse a query (e.g. a database connection) and some parser-related things (e.g. the reader) that might be the same all the time. What is the purpose of having something like the db connection in the environment argument? My connection is defined in another namespace and managed by mount and I've just been requiring it in the namespace where I define my mutations and accessing it directly as required rather than putting it in env and getting it out of there in my mutation. Is there a reason to do it another way?#2020-05-1416:35souenzzoIf db is defined on env, you can just assoc your test-db and run your test#2020-05-1419:18kszaboyeah, it’s the classic DI vs. global state question. I would argue that the DI approach is superior albeit it introduces more indirection#2020-05-1419:19kszabowhich makes @souenzzo’s case possible without restoring to mount’s own mocking system#2020-05-1420:09fjolne@ericihli there’s also a consistency consideration: env plugin injects the env on every parser call so that you have the same db value in all resolvers in that call#2020-05-1620:51yendais there a way to achieve union queries with a defresolver? (not sure if I'm using the right terminology here) Similar to this example https://cljdoc.org/d/com.wsscode/pathom/2.3.0-alpha6/doc/pathom-developers-guide#_union_queries but using defresolvers basically I have a resolver that returns a list of things as a result, with different types, and I'd like the result not to return with pathom-not-found for the fields that aren't required for a given type#2020-05-1709:00yendathe union-query example works with a parser but not with parallel-parser#2020-05-1709:28yenda
async-parser: {:search [{} {} {}]}
parallel-parser: {:search []}
parser: {:search [{:user/name Jack Sparrow} {:movie/title Ted} {:book/title The Joy of Clojure}]}
#2020-05-1712:23yendaSo far the only solution I found was to make a reader instead of a resolver:
(def user-notifications
  {:user/notifications
   (fn [env]
     (go-catch
        (<! (join-seq-parallel (assoc env
                                      ::p/union-path
                                      (fn [{:keys [query] :as env}]
                                        (let [e (p/entity env)]
                                          (:notification/type e))))
                               n)))))})
#2020-05-1712:23yendaand to redefine join-seq-parallel so that it doesn't check if the query is a vector#2020-05-1718:40wilkerlucio@yenda thanks for the report, looking into it now#2020-05-1720:07wilkerlucio@yenda fixed by: https://github.com/wilkerlucio/pathom/pull/161 going to release alpha-8 soon#2020-05-1720:13wilkerlucioalso add new docs on unions + connect: https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/resolvers.html#_union_queries#2020-05-1720:24yendanice thanks! There is no solution for when you have a :type key for instance, because there isn't really any field unique to the entity?#2020-05-1721:51wilkerlucioyou can still use that, by overwriting the ::p/union-path on the pathom env, this can be a (fn [env]) so you have full control over what branch to pick#2020-05-1811:20myguidingstaris there any function to extract the eql from an arbitrary piece of data, eg given {:x 1 :y 2 :z [{:t 3 :u 4}] returns [:x :y {:z [:t :u]}]?#2020-05-1815:02wilkerluciopc/data->shape#2020-05-1815:12myguidingstarmust be it. Thank you!#2020-05-1811:43kszabonot currently AFAIK#2020-05-1813:05dvingoI am connecting my pathom parser to pathom viz and I'm getting transit encoding errors for unsupported types, in this case #tick/date. Is there a place to add transit handlers for your own types?
#error {
 :cause "Not supported: class java.time.LocalDate"
 :via
 [{:type java.lang.RuntimeException
   :message "java.lang.Exception: Not supported: class java.time.LocalDate"
   :at [com.cognitect.transit.impl.WriterFactory$1 write "WriterFactory.java" 65]}
  {:type java.lang.Exception
   :message "Not supported: class java.time.LocalDate"
   :at [com.cognitect.transit.impl.AbstractEmitter marshal "AbstractEmitter.java" 194]}]
 :trace
 [[com.cognitect.transit.impl.AbstractEmitter marshal "AbstractEmitter.java" 194]
  [com.cognitect.transit.impl.JsonEmitter emitMap "JsonEmitter.java" 171]
  [com.cognitect.transit.impl.AbstractEmitter emitMap "AbstractEmitter.java" 85]
  [com.cognitect.transit.impl.AbstractEmitter marshal "AbstractEmitter.java" 184]
  [com.cognitect.transit.impl.AbstractEmitter emitArray "AbstractEmitter.java" 97]
  [com.cognitect.transit.impl.AbstractEmitter marshal "AbstractEmitter.java" 182]
  [com.cognitect.transit.impl.JsonEmitter emitMap "JsonEmitter.java" 171]
  [com.cognitect.transit.impl.AbstractEmitter emitMap "AbstractEmitter.java" 85]
  [com.cognitect.transit.impl.AbstractEmitter marshal "AbstractEmitter.java" 184]
  [com.cognitect.transit.impl.JsonEmitter emitMap "JsonEmitter.java" 171]
  [com.cognitect.transit.impl.AbstractEmitter emitMap "AbstractEmitter.java" 85]
  [com.cognitect.transit.impl.AbstractEmitter marshal "AbstractEmitter.java" 184]
  [com.cognitect.transit.impl.AbstractEmitter marshalTop "AbstractEmitter.java" 211]
  [com.cognitect.transit.impl.JsonEmitter emit "JsonEmitter.java" 41]
  [com.cognitect.transit.impl.WriterFactory$1 write "WriterFactory.java" 62]
  [cognitect.transit$write invokeStatic "transit.clj" 167]
  [cognitect.transit$write invoke "transit.clj" 164]
  [com.wsscode.transit$write invokeStatic "transit.cljc" 37]
  [com.wsscode.transit$write invoke "transit.cljc" 33]
  [_connector.impl.http_clj$send_message_BANG_ invokeStatic "http_clj.clj" 18]
  [_connector.impl.http_clj$send_message_BANG_ invoke "http_clj.clj" 15]
  [_connector.impl.http_clj$connect_parser$fn__56379 invoke "http_clj.clj" 75]
  [_connector.core$connect_parser$connected_parser__56389$fn__56445$state_machine__7701__auto____56470$fn__56473 invoke "core.cljc" 57]
  [_connector.core$connect_parser$connected_parser__56389$fn__56445$state_machine__7701__auto____56470 invoke "core.cljc" 56]
  [clojure.core.async.impl.ioc_macros$run_state_machine invokeStatic "ioc_macros.clj" 978]
  [clojure.core.async.impl.ioc_macros$run_state_machine invoke "ioc_macros.clj" 977]
  [clojure.core.async.impl.ioc_macros$run_state_machine_wrapped invokeStatic "ioc_macros.clj" 982]
  [clojure.core.async.impl.ioc_macros$run_state_machine_wrapped invoke "ioc_macros.clj" 980]
  [clojure.core.async.impl.ioc_macros$take_BANG_$fn__7719 invoke "ioc_macros.clj" 991]
  [clojure.core.async.impl.channels.ManyToManyChannel$fn__2538$fn__2539 invoke "channels.clj" 95]
  [clojure.lang.AFn run "AFn.java" 22]
  [java.util.concurrent.ThreadPoolExecutor runWorker "ThreadPoolExecutor.java" 1128]
  [java.util.concurrent.ThreadPoolExecutor$Worker run "ThreadPoolExecutor.java" 628]
  [clojure.core.async.impl.concurrent$counted_thread_factory$reify__2407$fn__2408 invoke "concurrent.clj" 29]
  [clojure.lang.AFn run "AFn.java" 22]
  [java.lang.Thread run "Thread.java" 834]]}
#2020-05-1815:06wilkerluciocan you check with transit version are you using? they had a bug that was recently fixed related to fallback encodings, I believe if you use latest transit it may fix, but if not please let me know and lets debug it#2020-05-1815:43dvingosure thing: using clj -Stree:
com.wsscode/pathom-viz-connector 1.0.2
  com.wsscode/async 1.0.4
  com.cognitect/transit-clj 1.0.324
    com.cognitect/transit-java 1.0.343
com.cognitect/transit-cljs 0.8.264
com.cognitect/transit-js 0.8.861
I have the following in deps.edn:
com.fulcrologic/fulcro {:mvn/version "3.2.5"
                         :exclusions  [com.cognitect/transit-cljs                       
                                      com.cognitect/transit-js]}
com.cognitect/transit-cljs          {:mvn/version "0.8.264"}
com.cognitect/transit-js            {:mvn/version "0.8.861"}
#2020-05-1815:45dvingothe cljs and js versions were older pulled in from fulcro, I updated those but the issue is still happening (i assume it's in clojure anyway not cljs)#2020-05-1821:16dvingoI'm still seeing the same error. I can try running the app locally in dev mode to see if i can find where it's falling over#2020-05-2000:23wilkerluciosorry the delay, got sidetracked on this, ok, I decided to force the fix to don't rely on transit version, I was able to run a parser returning a LocalDate, can you try version 1.0.3 of the connector and see if that works?#2020-05-2002:54dvingono problem! thanks for the quick fix, looks like it's working now šŸ™‚#2020-05-1815:02wilkerluciopc/data->shape#2020-05-1816:16lepistaneHello, I apologize in advance for the question for the zone "i dont know what i dont know" i am very interested in pathom but don't know how to handle it. it's just too complex and i lack foundational knowledge in order to use it. I couldn't get EQL but after looking at GraphQL and playing around with it little bit with my brother when he was preparing for interview i finally understood EQL. I even get pathom bit more but still have issues even with simple example from docs. Where would i need to look to build up my foundation and better understand terminology and thinking about this topic? I feel like with pathom i could create integration between api and this seems very very powerful#2020-05-1821:03dvingoI'd say one of the best stories for learning is focusing on fulcro, which makes using pathom easier. There's tons of documentation and videos. https://www.youtube.com/playlist?list=PLVi9lDx-4C_T7jkihlQflyqGqU4xVtsfi#2020-05-1821:03dvingoSome of the server videos in that playlist have pathom setup#2020-05-1820:52souenzzoAs a early-user of om/fulcro/pathom/EQL, the community is really small and there is a lot of WIP things (including naming) We need more users, more discussion, more uses to advance in naming/docs/explanations#2020-05-1821:09wilkerluciowe gonna a real good oportunity for that soon, stay tuned šŸ™‚#2020-05-1919:44souenzzo@wilkerlucio is it a "undefined" behavior? If there is 2 resolvers to resolve :a, one is "global", the other need a input :i The pc/reader3 will prefer "the first one", where "the first one" is "the first inserted in the index" (aka the register order matters, with I think is bad) Sample case
(let [register [(pc/resolver
                  `user-name-by-id
                  {::pc/input  #{:user/id}
                   ::pc/output [:user/name]}
                  (fn [_ {:user/keys [id]}]
                    {:user/name (str "user-" id)}))
                (pc/resolver
                  `current-user-name
                  {::pc/output [:user/name]}
                  (fn [_ _]
                    {:user/name "current"}))]
      parser-1 (p/parser {::p/plugins [(pc/connect-plugin {::pc/register register})]})
      parser-2 (p/parser {::p/plugins [(pc/connect-plugin {::pc/register (reverse register)})]})
      env {::p/reader               [p/map-reader
                                     pc/reader3
                                     pc/open-ident-reader
                                     p/env-placeholder-reader]
           ::p/placeholder-prefixes #{">"}}]
  {:parser-1 (get (parser-1 env [{[:user/id 1] [:user/name]}]) 
                  [:user/id 1])
   :parser-2 (get (parser-2 env [{[:user/id 1] [:user/name]}])
                  [:user/id 1])})
=> {:parser-1 {:user/name "user-1"}, :parser-2 {:user/name "current"}}
#2020-05-1920:05wilkerluciocurrently reader3 doesn't have a real algorithm to pick a path, the current impl will use first, that's a current incomplete part of it#2020-05-1920:12souenzzoIs it tracked? Should I open something to track it? I'm not sure if how it should behave actually#2020-05-1920:45markaddlemanI've never used Pathom before but it seems like a natural fit for my usecases to access a bunch of data in ElasticSearch. One of usecases is to serve autocomplete in the UI. I don't see any reason NOT to use Pathom for this usecase (except, possibly, performance). Am I wrong?#2020-05-1923:51wilkerlucio@markaddleman in the end pathom is just a glue to compose information across your system, I have used pathom with ES before, don't see any reason not to šŸ™‚#2020-05-1923:51markaddlemanThanks for confirming! I'm looking forward to this little project#2020-05-2001:27Chris O’DonnellWe're planning on putting pathom into production (server-side) at work soon. Any tips from those who have production experience?#2020-05-2010:17souenzzoWill be useful be abre to connect pathom-viz with trace support#2020-05-2003:59tomjkiddFor an eql query where all the keys refer to clojure specs, is there a handy way to leverage pathom to produce a response using those specs?#2020-05-2010:18souenzzoThere is a pathom.generator or pathom.gen namespace. It still binded with fulcro but you can use "as inspiration"#2020-05-2011:24tomjkiddThank you, I’ll take a look šŸ™‚#2020-05-2119:50Chris O’Donnell@wilkerlucio is it possible to inject something into the parser env from pathom-viz? My eql endpoint adds some authentication data into env that my resolvers need, and I'd like to simulate that.#2020-05-2120:06wilkerlucionot sure if I understand, the pathom-viz should be using your parser at all the times, all the calls always goes though your parser, so you shouldn't need to simulate things, are you using the standalone app with the connector?#2020-05-2120:09Chris O’DonnellYeah, I was thinking about the case of sending queries from the standalone app.#2020-05-2120:09wilkerluciowhen you do that, it is still processing on your real parser#2020-05-2120:10wilkerluciopathom-viz itself doens't have a parser (to be honest, it does, but its purely for UI purposes, nothing of that is related to calling your parser)#2020-05-2120:10wilkerluciois this something you are experiencing or you were having thoughs about it?#2020-05-2120:10Chris O’DonnellSo would you recommend doing something like conditionally adding fake authentication data into the env in a dev-only env wrap plug in?#2020-05-2120:10Chris O’DonnellI am experiencing it.#2020-05-2120:10wilkerluciocompared to calling the parser directly, the diff is that you can't add extra env, so all the parser configuration needs to be there already#2020-05-2120:11wilkerlucioyeah, you can do that, and consider that the parser can be wrapped, so you can do something like this:#2020-05-2120:12wilkerlucio
(def tracked-parser
    (p.connector/connect-parser
      {::p.connector/parser-id ::my-parser}
      (fn [env tx]
        (parser (assoc env :extra-connector-stuff "bla") tx))))
#2020-05-2120:13Chris O’DonnellAh that's a bit simpler. Thanks!#2020-05-2120:21Chris O’DonnellI'm currently working on replacing a somewhat crufty lacinia-based graphql server with a pathom-backed eql endpoint, and pathom has really been sparking joy. Thanks so much for all the work you've put in!#2020-05-2214:03markaddlemanI'm on my pathom learning journey and am running into a problem that makes me think I'm missing some fundamental concept. In the docs, there's the latest-product resolver. I want to extend that to take an input and return a list of products. For example, I want to provide a store id and return the list of latest products for that store. Here are the resolver and query that don't work. What am I doing wrong?
(pc/defresolver latest-products-for-store [_ {::keys [store]}]
  {::pc/input  [::store]
   ::pc/output [{::store [:product/id :product/title :product/price]}]}
  (condp = store
    0 {[::store 0] [{:product/id    1
                     :product/title "Acoustic Guitar"
                     :product/price 199.99M}
                    {:product/id    2
                     :product/title "Piano"
                     :product/price 99.99M}]}
    1 {[::store 1] [{:product/id    3
                     :product/title "Clavachord"
                     :product/price 199.99M}
                    {:product/id    4
                     :product/title "Didgeridoo"
                     :product/price 990.99M}]}))
query:
[{[::store 1] [:product/id]}]
#2020-05-2214:06markaddlemantypo in my example the ::pc/input ought to be a set instead of a vector. That doesn't change the result. The parser still returns not found#2020-05-2214:50markaddlemanI found the issue: The output attribute is the same name as the input attribute (ie ::store) Of course, that makes no sense#2020-05-2216:40dvingonice find, I've struggled with typos and keys not matching. Maybe the defresolver macro can be enhanced to support some basic type checking like that#2020-05-2306:08currentoorIs it wrong to invoke the parser in a resolve? What about multiple times?#2020-05-2306:09currentoorI’m getting weird behavior and starting to think this is not how the parser is supposed to be used.#2020-05-2306:13currentoorIf I invoke the parser multiple times with pmap rather than map it works#2020-05-2318:13currentoorIs it ok to bust the cache in the env if I want to invoke the parser from within a resolver?#2020-05-2322:12wilkerlucioyeah, calling from inside should be fine, but can lead to unexpected situations due to caching. the best you can do is try and see what works for the situation#2020-05-2421:34currentoori reset the atoms in the env (there were three of them pathom had) and now it works great#2020-05-2421:34currentoorthanks!#2020-05-2415:51markaddlemanI'm having trouble with query parameters. My query is
'[{([:a "value"] {:with "params"})
                [:b :c]}]
Which, I believe matches the syntax for ident joins described at https://edn-query-language.org/eql/1.0.0/specification.html#_parameters The ast does not include the parameter map
#2020-05-2415:53markaddlemanThe ast for query
'[{[:a "value"]
              [(:b {:with "params"}) :c]}]
includes parameters but I don't think this syntax matches the spec. From what I understand, I also don't think this query makes sense
#2020-05-2415:56markaddlemani'm using pathom 2.3.0-alpha8#2020-05-2416:33dvingoyou can see how your queries get parsed with eql lib:
(eql/query->ast
    '[{([:a "value"] {:with "params"})
       [:b :c]}]
    )
=>
{:type :root,
 :children [{:type :join,
             :dispatch-key :a,
             :key [:a "value"],
             :params {:with "params"},
             :meta {:line 2, :column 8},
             :query [:b :c],
             :children [{:type :prop, :dispatch-key :b, :key :b} {:type :prop, :dispatch-key :c, :key :c}]}]}
#2020-05-2416:43markaddleman
[{([:a "value"] {:with "params"})
                [:b :c]}]
#2020-05-2416:43markaddlemanThe resolver does not receive the params for that ^^ query#2020-05-2416:45markaddlemanThat query's ast is
{:type :prop, :dispatch-key :b, :key :b}
#2020-05-2416:49dvingook I see. It will be hard to debug without your resolver code and maybe an example of invoking the parser and logging what the resolver's input and (:ast env) are#2020-05-2416:51markaddlemanHere's a repro case:
(pc/defresolver param-resolver [env _]
  {::pc/input  #{:a}
   ::pc/output [:b :c]}
  (clojure.pprint/pprint (:ast env))
  {:b 1, :c 2})

(def parser
  (p/parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/reader2
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate-async
     ::p/plugins [(pc/connect-plugin {::pc/register [param-resolver]}) ; setup connect and use our resolvers
                  p/error-handler-plugin
                  p/trace-plugin]}))

(parser {} '[{([:a "value"] {:with "params"})
              [:b :c]}])
#2020-05-2416:52markaddlemanI'm sure there's a pathom plugin to log the ast parsing but my pathom foo isn't up to it šŸ™‚#2020-05-2417:14dvingook I tried it out and I'm not seeing the params coming through to the resolver either. They are in the transaction though, before being processed by the resolver. I'm not sure why that is, but to get this working you can use the plugin in fulcro-rad to pass params as a top level key :query-params in the env: https://github.com/fulcrologic/fulcro-rad/blob/develop/src/main/com/fulcrologic/rad/pathom.clj#L108 Then pull them off any resolver using (:query-params env)#2020-05-2417:17markaddlemanOk, so I'm a pathom newbie and I'm not using fulcro. From my understanding, I don't think the plugin from fulcro-rad has any dependency on fulcro - so that's good. Does it the ordering of the rad plugin matter relative to the connect plugin?
#2020-05-2417:18dvingonp, you can just copy paste it šŸ™‚#2020-05-2417:19dvingoi'm not sure about ordering with plugins, but i have it after the connect plugin#2020-05-2519:23markaddlemanI have a pathom client that speaks json and I'd rather not have to get it to speak edn. Does anyone know of a json form of eql that has a straightforward translation to eql?#2020-05-2519:25markaddlemanI'm particularly concerned about representing idents and eql parameters.#2020-05-2519:32myguidingstarwhat do you mean by "pathom client that speaks json"?#2020-05-2519:34souenzzoI have a lib called eql-as that I use for it. Checkout "RealWorld" example In the product that I work on, we use to map a REST API, with unqualified keys, both GET and POST's to EQL queries then dispatch it for a parser that only knows about qualified keys https://github.com/souenzzo/eql-as#2020-05-2519:37markaddlemanI mean, my client is a typical browser-based javascript application. I want it to make queries that would be resolved by my pathom server-side application#2020-05-2519:38markaddleman@U2J4FRT2T Thanks! The library looks like it will be helpful after I come up with a json syntax for eql. My challenge right now is to come up with that syntax! šŸ™‚#2020-05-2519:45souenzzoThis lib still incomplete. any kind of feedback is welcome šŸ™‚#2020-05-2519:29myguidingstarI want to create an async version of this parser so it can run under nodejs https://github.com/walkable-server/walkable/blob/3b1c218ef60c90b7a79f968c87e2ee0af46608b3/dev/src/common/walkable/integration_test/helper.clj#L27#2020-05-2519:30myguidingstarit uses planner, btw#2020-05-2519:30myguidingstarwhat do i have to change beside the dynamic resolver function?#2020-05-2519:34myguidingstareg: change p/parser to p/parallel-parser, pc/reader3 to something...#2020-05-2521:58markaddlemanIt looks like pathom does an outer join when the results of two resolvers are joined together. Is there a way to get it to do an inner join instead?#2020-05-2601:52wilkerlucionot sure what you mean, can you give an example?#2020-05-2603:17markaddleman
(ns com.wsscode.pathom.book.connect.getting-started2
  (:require [com.wsscode.pathom.core :as p]
            [com.wsscode.pathom.connect :as pc]))


(pc/defresolver people-resolver [_ _]
  {::pc/input  #{}
   ::pc/output [{:people [:name]}]}
  {:people [{:name "Mark"}
            {:name "Joe"}]})

(pc/defresolver address-resolver [_ {:keys [name]}]
  {::pc/input  #{:name}
   ::pc/output [:address]}
  (if (= name "Mark")
    {:address "111 Main St"}))

(def app-registry [people-resolver
                   address-resolver
                   pc/index-explorer-resolver])

(def parser
  (p/parser
    {::p/env     {::p/reader [p/map-reader
                              pc/reader2
                              pc/open-ident-reader]}
     ::p/mutate  pc/mutate
     ::p/plugins [(pc/connect-plugin {::pc/register app-registry})
                  p/error-handler-plugin
                  p/trace-plugin]}))

(parser {} [{:people [:name :address]}])
#2020-05-2603:18markaddlemanThe result is
{:people [{:name "Mark", :address "111 Main St"} {:name "Joe", :address :com.wsscode.pathom.core/not-found}]}
I'd like to filter out Joe
#2020-05-2603:28markaddlemanI suspect that I have to write a plugin to do this. Am I on the right track?#2020-05-2603:36markaddlemanAfter looking at the plugin architecture, I think that's the wrong approach. Perhaps post-processing the results is the right way#2020-05-2601:52wilkerlucio@myguidingstar I suggest you use the async-parser for that#2020-05-2605:25jacklombardHey all, how does HTTP/CDN caching work for pathom/transit?Ā Can it be done? If yes, is purging the cache straight forward?#2020-05-2610:12Chris O’DonnellIf you wanted to do http caching of eql queries, you could try passing the eql as a query param. Pathom doesn't specify how eql gets from your client to the server. That is completely up to you.#2020-05-2606:04myguidingstar@wilkerlucio my question is what else I need to change to use async, given that I use (-> env ::pcp/node) in my dynamic resolver#2020-05-2607:42wilkerluciothat could be a bug, does it work normally on sync parsers but not on async?#2020-05-2608:06myguidingstaryes, it works on sync#2020-05-2606:06myguidingstarI tried p/parallel-parser and async-parser, but ::pcp/nodereturns nil in both cases, so I guess there's something I miss#2020-05-2608:10myguidingstarmy bad, I pass the wrong order of params to resolve function#2020-05-2608:10myguidingstar(-> env ::pcp/node)does work for async parser#2020-05-2614:20Jakub Holý (HolyJak)@wilkerlucio FYI Import of latest v 1.0.10 to cljdoc failed https://cljdoc.org/builds/31120#2020-05-2615:51Michael WAre there any good examples of using the parser? I'm still new to clojure, got my api data sources all wired up into pathom, and now I can't seem to figure out how no run multiple queries and pull the data out. I can do 1, and I get the data in a PersistentArrayMap but I'm kinda lost at that point...#2020-05-2616:18Chris O’Donnell@michael819 you could take a look at pathom's tests.#2020-05-2616:19Michael WI have been looking at those, but when I run a query I get a map back with the query as the key, but in those tests it looks like there is no key, just a map of results#2020-05-2616:20Chris O’DonnellAlso perhaps a little less involved: https://chrisodonnell.dev/posts/giftlist/parser_tests/#2020-05-2616:20Chris O’DonnellHow are you calling the parser?#2020-05-2616:21Michael W
(parser {} [{[:ddi/ip "192.168.1.111"] [:ddi/name :ddi/mac :aci/mac]}])
{[:ddi/ip "192.168.1.111"] {:ddi/name "", :ddi/mac "00:25:b5:21:a0:57", :aci/mac "00:25:b5:21:a0:57"}}
#2020-05-2616:22Chris O’DonnellThat looks right to me#2020-05-2616:22Chris O’DonnellThe key in your result map is the ident for which you made the query#2020-05-2616:22Chris O’DonnellAnd the value has all the attributes you queried for#2020-05-2616:22Michael WYeah, but in the tests for pathom his = assertions all just have the results, without the ident key#2020-05-2616:25Chris O’DonnellCan you link to a specific test?#2020-05-2616:25Michael W
(is (= (parser {} [:foo])
           {:foo "bar"}))

    (is (= (parser {} [:foo :aa])
           {:foo "bar"
            :aa  :not-found}))
#2020-05-2616:26Michael WFrom here: https://github.com/wilkerlucio/pathom/blob/master/test/com/wsscode/pathom/parser_test.clj#2020-05-2616:27Michael WAll the assertions are just for the results, there is no map key for the ident, which is what confused me.#2020-05-2616:30Michael WThe link you provided is much clearer, I see you using select-keys and get to pull the results out using the ident.#2020-05-2616:41Chris O’DonnellThose tests aren't using pathom connect, which is why the result structure is different.#2020-05-2616:44Chris O’DonnellI won't recommend looking at pathom's tests in the future; they aren't super helpful for a beginner, it seems! šŸ˜…#2020-05-2616:45Chris O’DonnellEven the connect tests are mostly verifying internals or seem to be constructed to verify certain bits of behavior rather than to illustrate how the parser is meant to be used.#2020-05-2616:53Michael WI thought the developer guide was good except that one thing, everything in there doesn't show how to really use the parser.#2020-05-2617:02Chris O’Donnell@michael819 Top level keys like that are another way to use the parser. In my experience they're usually used to get to a set of entities, but they can also be used as global constants (or to do something like inject the current timestamp) as they seem to be in the test you linked. You could reproduce the parser from your test with this resolver:
(defresolver foo-resolver [_ _]
  {::pc/input #{}
   ::pc/output [:foo]}
  {:foo "bar"})

(def parser ...)

(parser {} [:foo])
{:foo "bar"}
Here's an actual resolver from the project the blog post I linked above was talking about that also uses a top level key:
(defresolver created-gift-lists
  [{::db/keys [pool] :keys [requester-id]} _]
  {::pc/input #{}
   ::pc/output [{:created-gift-lists [::gift-list/id]}]}
  {:created-gift-lists
   (db/execute! pool
     {:select [:gl.id]
      :from [[:gift_list :gl]]
      :where [:= requester-id :gl.created_by_id]
      :order-by [:gl.created_at]})})

(def parser ...)

(parser {:requester-id 1} [{:created-gift-lists [::gift-list/id]}])
{:created-gift-lists [{::gift-list/id 1} {::gift-list/id 2}]}
#2020-05-2617:04Chris O’DonnellI'm sure Wilker talks about this in at least one of his talks. Maybe this one? https://www.youtube.com/watch?v=IS3i3DTUnAI#2020-05-2617:09myguidingstar@wilkerlucio I have this piece of code that works on clj but somehow slightly incorrect in cljs https://gist.github.com/myguidingstar/6bff072b099cc0d5a178c61f828eb925#2020-05-2617:12myguidingstarI guess it's a bug with async planner in cljs#2020-05-2912:52jeroenvandijkAnyone aware of a Pathom resolver that allows to walk the file system or a git repo?#2020-05-2912:59myguidingstarI've heard of none, but what's the use case? Putting a pathom interface in front of things may or may not add any value#2020-05-2912:59jeroenvandijkIt’s a very specific one šŸ™‚#2020-05-2913:00jeroenvandijkI want to be able to browse files in my new web framework haha#2020-05-2913:00jeroenvandijkI want to do this without leaking implementation details in my view (the whole reason of using Pathom I believe)#2020-05-2913:01jeroenvandijkSo i was thinking of starting with something simple like a file browser#2020-05-2913:01jeroenvandijkMaybe someone else was crazy enough to do this as well#2020-05-2913:01jeroenvandijkI guess not šŸ™ˆ#2020-05-2914:00jeroenvandijkAny feedback on this? I want it to work recursively, but I’m still trying to get that to work#2020-05-2914:05wilkerlucio@jeroenvandijk hello, first I like to suggest you to use the simple parser instead of parallel, will be faster for this case. And I guess looking at the code that you can do recursive calls already, you can try this query:
(clojure.pprint/pprint
 (async/<!! (parser {::p/entity {:file/path "."}}
                    '[:file/path
                      :file/type
                      {:dir/files ...})))
#2020-05-2914:07jeroenvandijkthank you! It’s not resolving the :dir/files recursively. I think I should add some join functionality somewhere. Reading that documentation now, but it didn’t click yet for me#2020-05-2914:11jeroenvandijkThis is the output. I would expect/hope ./src and ./git to be expanded
{:file/path ".",
 ... nil,
 :file/type :dir,
 :dir/files
 [#:file{:path "./project.clj"}
  #:file{:path "./.gitignore"}
  #:file{:path "./.nrepl-history"}
  #:file{:path "./.git"}
  #:file{:path "./src"}]}
#2020-05-2914:14wilkerlucio@jeroenvandijk I wrote my version of it, this works:
(pc/defresolver file-resolver [env {:file/keys [path]}]
  {::pc/input  #{:file/path}
   ::pc/output [:file/type]}
  (let [f    ( path)
        dir? (.isDirectory f)]
    {:file/type (if dir? :dir :file)
     :file/dir? dir?}))

(pc/defresolver directory-files-resolver [env {:file/keys [path dir?]}]
  {::pc/input  #{:file/path :file/dir?}
   ::pc/output [{:dir/files [:file/path]}]}
  (if dir?
    {:dir/files
     (mapv (fn [^File f0] {:file/path (.getPath f0)})
       (.listFiles ( path)))}))

(def my-resolvers [file-resolver directory-files-resolver])

(def parser
  (p/parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/reader2
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate
     ::p/plugins [(pc/connect-plugin {::pc/register my-resolvers})
                  p/error-handler-plugin
                  p/request-cache-plugin
                  p/trace-plugin]}))

(comment
  (parser {} [{[:file/path "src"]
               [:file/path
                :file/type
                {:dir/files '...}]}]))
#2020-05-2914:17jeroenvandijkwow @wilkerlucio thank you!#2020-05-2914:18jeroenvandijkSo I guess my version didn’t trigger the recursion because :dir/files was already in the returning map, right?#2020-05-2914:19wilkerlucioI don't think that's the issue, your code seems like it should work#2020-05-2914:19wilkerluciobut your initial query example was off#2020-05-2914:20wilkerluciowas missing the map on the join for :dir/files#2020-05-2914:20wilkerlucio:dir/files [* {:dir/files [*]}]#2020-05-2914:20wilkerlucioā˜ļø missing {} around#2020-05-2914:21jeroenvandijkAh thank you. Good to understand this šŸ™‚#2020-05-2914:23jeroenvandijkyou are right, my query was wrong. The resolver was working#2020-05-2914:29jeroenvandijkI guess it is in general a best practise to make your resolvers as small as possible. For reusability and maybe also for performance?#2020-05-2914:34jeroenvandijkFrom the docs:
Pathom will scan through the defined resolvers in order to try to satisfy all of the properties in a query. So, technically you can split up your queries as much as makes sense into separate resolvers, and as long as the inputs are in the context Pathom will assemble things back together.
I guess my resolver was indeed not lazy enough
#2020-05-2914:35jeroenvandijkThanks @wilkerlucio. You have created an amazing library. Looking forward to use it more#2020-05-2914:51jeroenvandijkIs it possible to have context dependent resolvers? E.g. when I want to browse a git tree {:dir/files …} would need to trigger a different resolver than when I browse a normal file system. I could also use a different key for that, but it would require more knowledge of the user#2020-05-2914:53jeroenvandijkI think I have an idea šŸ™‚#2020-05-2914:56jeroenvandijkI think this will work when I implemented
(pc/defresolver file-resolver [{:keys [:resolvers.file/root-dir] :as env} {:file/keys [path]}]
  {::pc/input  #{:file/path}
   ::pc/output [:file/type]}
  (let [f    ( (str root-dir) path)
        dir? (.isDirectory f)]
    {:file/type (if dir? :dir :file)
     :file/dir? dir?
     :filesystem/type :normal}))


(pc/defresolver directory-files-resolver [{:keys [:resolvers.file/root-dir] :as env} {:file/keys [path dir?] filesystem-type :filesystem/type}]
  {::pc/input  #{:file/path :file/dir? :filesystem/type}
   ::pc/output [:dir/files]}
  (let [relativy (if root-dir
                   (let [idx (count root-dir)]
                     (fn [path]
                       (subs path idx)))
                   identity)]
    (if (= filesystem-type :normal)
      {:dir/files
       (when dir?
         (mapv (fn [^java.io.File f0] {:file/path (relativy (.getPath f0))})
               (.listFiles ( root-dir path))))}
      (throw (ex-info "not implemented yet" {}))
      )))
When I write a different resolver for a git filesystem the user doesn’t have to know it’s querying something different šŸ™‚
#2020-05-2915:26wilkerlucio@jeroenvandijk you can just make multiple resolvers for the same key as well, and in the case out of context you return nil on the resolver, so pathom will try the other one#2020-05-2915:27jeroenvandijkAh awesome!#2020-05-2915:54Reily SiegelHello, is there an acceptable way to access the parser from within a mutation? I have two use cases for this: 1. Resolve unknown information that a mutation needs to function, that the caller of the mutation does not know. 2. Have a mutation submit several other "sub-mutations" to make smaller, reusable mutations For context, my problem domain is interfacing with the Discord API. So for example, for a higher level feature, I might need to make several "sub-mutations" (create post, add a reaction, etc). I also dont want to burden callers of the mutation with knowing details. For example, instead of requiring the "create post" mutation to accept a :discord.channel/id parameter, it would be more convenient to accept a :discord.channel/name and resolve the id based on an existing resolver. Am I thinking about this completely wrong?#2020-05-2915:55myguidingstar@reilysiegel there's :parser key in env already#2020-05-2915:58Reily SiegelThank you! So in that case the solution to 2 is obvious, and for 1 I could pass the parameter map as the context to the resolver, and the desired parameters as a query.#2020-05-2915:58Reily SiegelIn fact, I could probably write a transform that automatically does that based on ::pc/params#2020-05-2916:02myguidingstara note to consider sub-mutations vs one composite mutation if you're using fulcro: each individual mutation may receive some data from server so your client db is updated - all the updates may be hard to denoted with a composite mutation#2020-05-2916:03myguidingstaralso, sub-mutations can "co-ordinate" by using tempids#2020-05-2916:04myguidingstaranyway, both ways are valid, just evaluate the tradeoffs#2020-05-2917:04kennyI was going to try out the latest pathom release 2.3.0-alpha9. Upon updating to that release from 2.2.30, I receive an exception trying to load a ns that uses pathom:
Syntax error (ClassNotFoundException) compiling at (com/fulcrologic/guardrails/config.cljc:17:1).
com.google.javascript.jscomp.CompilerOptions
From the changelog I saw 2.3.0-alpha4 bumped the dep that appears to be causing this issue - guardrails. I tried running with 2.3.0-alpha4 instead and received the same issue. I went all the way back to 2.3.0-alpha1 and still get the exception.
#2020-05-2917:23myguidingstar@kenny just a generic workaround: have you tried :exclusions in your dependencies?#2020-05-2917:24kennyWasn’t worth the time to try it out. That’s probably the solution though. One of pathom’s deps updated and now conflicts with something else. #2020-05-2917:56wilkerlucio@kenny I wonder if has something to do with cljs versions itself, are you running on a recent one?#2020-05-2918:03kennyThis isn't on a cljs app. It's a dep on a Clojure backend service.#2020-05-2918:04kennyI suppose cljs could be brought in accidentally.#2020-05-2918:12wilkerluciomaybe, what makes me think around that is because the error seems go be around some missing google closure thing#2020-05-2918:12wilkerlucioand from the deps jump you did, guardrails is a new dep#2020-05-2918:12kennyThe line of code the exception is pointing to also has this comment:
;; This isn't particularly pretty, but it's how we avoid
;; having ClojureScript as a required dependency on Clojure
#2020-05-3116:01Andreas EdvardssonI'm trying to get a union resolver/query to work but have failed so far. I have made a simple example with my best attempt so far, and I would be really happy if someone could point out what I'm doing wrong šŸ™‚ Is it correct that union resolvers are specified just as a normal output join, and the union "logic" is present just in the query? I'm on pathom version 2.2.30.#2020-06-0108:25Andreas EdvardssonI think I've figured out what I'm doing wrong now, union resolvers can't be "to-many" (or can they?).#2020-06-0108:32Andreas EdvardssonBy introducing the concept of a single "search-result", the resolver now looks like:
(pc/defresolver search-resolver
  [_ {:keys [search/term]}]
  {::pc/input  #{:search/term}
   ::pc/output [{:search/results
                 [{:search/result [:image/id :video/id]}]}]}
  {:search/results [{:search/result {:image/id 1}}
                    {:search/result {:video/id 3}}]})
The query:
(async/<!! (parser {} [{[:search/term "my-search-term"]
                          [{:search/results
                            [{:search/result
                              {:image/id [:image/name]
                               :video/id [:video/title]}}]}]}]))

  ;; => {[:search/term "my-search-term"] #:search{:results [#:search{:result #:image{:name "Red flower"}}
  ;;                                                        #:search{:result #:video{:title "A day in life"}}]}}
Success!
#2020-06-0102:28markaddlemanMy data is naturally hierarchical: events have attributes and attributes have attributeValues And the user walks that hierarchy in the UI by first select an event, then attribute and then an attribute value. It seems like I should be able to construct a pathom query like this
[{[:event "an event"] [{[:attribute "some attribute"] [:attributeValues]}]}]
But this resolver doesn't get invoked:
(pc/defresolver event-attribute-values [env input]
  {::pc/input  #{:event :attribute}
   ::pc/output [{:attributeValues [:attributeValue]}]}
  (println "event-attribute-values" input)
  {:attributeValues (into [] (select-keys-distinct [:attributeValue])
                          (es/list-result! [:attribute-values-v1] env input))})
#2020-06-0102:30markaddlemanI read in the docs that I can provide a pathom/context map but that seems contrary to the idea of a graph query.#2020-06-0102:56markaddlemanI think it's interesting that this query resolves:
[{[:event "Sign In Forgot Password Submitted"] [{:attributes [:attribute :path :attributeValues]}]}]
#2020-06-0102:57markaddlemanI expect that I could provide an ident in the place of :attributes#2020-06-0102:57markaddlemanAm I thinking of pathom wrong?#2020-06-0106:04Andreas EdvardssonI’m not very skilled in Pathom but what happens if you put all the context together?:
[{[:event "an event"
   :attribute "some attribute"] [:attributeValues]}]
#2020-06-0114:25markaddlemanThanks but that doesn't work either because an ident is defined as a vector with two elements.#2020-06-0114:25markaddlemanI did play around with a new reader that accepts a map as an ident instead of a two-element vector. It seems to work but I have more playing to do šŸ™‚#2020-06-0119:40sergey.shvetsPathom won't pass your :event attributes to the second join automatically. You need to do it manually. Just make sure that inside your :event resolver you pass down the :event property. You need to modify your event-attribute resolver to return :event that I assume it gets as an input.#2020-06-0119:41sergey.shvetsI now do it for every "join" resolver and it allows me for pretty much unlimited nested joins#2020-06-0119:43markaddlemanThanks for the insight. I'm pretty new to pathom - Can you give me an example of passing the event attribute to the second join?#2020-06-0119:44markaddlemanDo you mean that it's the responsibility of the resolver?#2020-06-0119:47markaddlemanI updated my resolver to pass along :event and now it works like I expect. Thanks!#2020-06-0120:06sergey.shvetsYes. I would vote for having this done by pathom, but for now it is responsibility of a resolver to do so.#2020-06-0120:16markaddlemanYeah, I agree with your vote šŸ™‚#2020-06-0120:17markaddlemanI ran into a related problem:#2020-06-0120:18markaddlemanI have two resolvers one with output
[{:attributes [:attribute :attributeType :dataType :path :event]}]
and another with output
[{:attributes [:attribute :attributeType :dataType :path]}]
(the only difference is that one produces :event and the other doesnot
#2020-06-0120:19markaddlemanThis reflects two different locations of :attributes in my graph: one set of attributes is global for the entire UI while the other set of attributes is event-specific#2020-06-0120:20markaddlemanThis query gets handled by the event-specific resolver:
{:events [:event {:attributes [:attribute :ui-location :dataType :path :attributeValues]}]}
#2020-06-0120:20markaddlemanBut this one gets resolved by the global resolver:
{[:event "My Savings Coupon Edited"] [{:attributes [:attribute :ui-location :dataType :path :attributeValues]}]}
#2020-06-0120:21markaddlemanThe obvious solution is to have two separate attribute lists, :global-attributes and :event-attributes#2020-06-0120:25sergey.shvetsyes, I think this is the best way to solve it. Alternatively, you can pass the proper resolver type as a param, but that is way more uglier, imo#2020-06-0120:28markaddlemanI agree#2020-06-0120:28markaddlemanThanks for the pointers!#2020-06-0120:38sergey.shvetsYou welcome. I'm pretty new to pathom as well, but Wilker was super helpful and helped me solve the exact same problem couple of weeks ago#2020-06-0413:30cldwalkerMornin. Latest api pathom docs are working again https://github.com/cljdoc/cljdoc/issues/400. There's also a hidden rebuild button on cljdoc that Martin pointed out#2020-06-0415:14wilkerlucionice! šŸ˜„#2020-06-0413:56Bjƶrn EbbinghausIs there someone who had a problem with datahike not writing to its file when transacting inside a parallel parser?#2020-06-0415:40Thomas MoermanHi, newbie question: Imagine you have a mutation in which some complex context data structure must be checked first, is it an idiomatic approach to use the existing resolvers (e.g. via the parser) to query for the needed context using an EQL query? (hope this makes sense)#2020-06-0415:42wilkerluciohello, you mean like, you have an input, then you use it compute more data, to then run the mutations, is like this?#2020-06-0416:06Thomas MoermanYes. My idea was that I could perhaps leverage all the resolver logic that is already implemented to produce contextual data (needed in the mutation) in function of the mutation params and a graph query.#2020-06-0416:07Thomas Moermanthen run some inspections on that data map to decide e.g. whether the mutation is allowed or not#2020-06-0513:14wilkerlucioyes, that's totally possible, I remember writing a transform fn in the past that did just that#2020-06-0513:14wilkerluciotrying to find it here#2020-06-0513:37wilkerluciocouldn't find, but in a basic sense, you can just call the parser from inside the mutation (use the one from env, there is a :parser key there)#2020-06-0616:53TuomasI’m looking for tips on what docs/resources to check for more info. I’m getting inconsistent results with paraller-parser. I have avoided parallel stuff by building resolvers that give all the data that client queries need in one resolvers. Now I have a case where I output something and it get’s used as input in another resolver. Everything works consistently in the repl, but fulcro client queries get inconsistent responses (for identical transit-params). I mostly get not-found for the entities that are resolved in parallel but sometimes I get the same response as I get in the repl.#2020-06-0618:02dvingoWhat does your vector of readers in ::p/reader look like? This parallel setup is working for me with a fulcro app:
{::p/mutate
 pc/mutate-async
 
 ::p/env
 {::p/reader
  [p/map-reader
   pc/parallel-reader
   pc/open-ident-reader
   p/env-placeholder-reader]
  
  ::p/placeholder-prefixes
  #{">"}}}
#2020-06-0703:44TuomasI’ll try that combination too and after that I’ll try to usee a serial parser
(p/parallel-parser
    {::pp/timeout-reach 60000
     ::p/env            {::p/reader                 [p/map-reader
                                                     pc/parallel-reader
                                                     pc/open-ident-reader
                                                     pc/reader2
                                                     pc/async-reader2
                                                     p/env-placeholder-reader]
                         ::pc/mutation-join-globals [:tempids]
                         ::pc/thread-pool           (pc/create-thread-pool (async/chan 4000))
                         ::p/placeholder-prefixes   #{">"}
                         ::p/process-error          (fn [env err] (println err) (p/error-message err))}
     ::p/mutate         pc/mutate-async
     ::p/plugins        [(pc/connect-plugin {::pc/register resolvers})
                         (p/env-wrap-plugin (fn [env]
                                              (let [conn (app.database/get-connection)]
                                                (assoc env
                                                  :db (d/db conn)
                                                  :conn conn))))
                         (p/post-process-parser-plugin (fn [input]
                                                         (let [output (p/elide-not-found input)]
                                                           (pprint output)
                                                           output)))
                         p/error-handler-plugin]})
#2020-06-0619:35wilkerlucio@koivistoinen.tuomas my general recommendation is to not use parallel parser, unless you can take real benefit from it, and I guess most users don't need it. parallel parser is way more complex than the others, and adds significant overhead, so unless you are running quite large queries that spend most time on IO, the serial parser is probably a better option, will be easier to understand, easier to debug, and probably faster in most cases#2020-06-0811:17Bjƶrn EbbinghausHm. And I thought the parallel parser was the recommended way…#2020-06-0705:48TuomasI found the bug. I was querying entities on 3 levels (simplified example) [:a/id {:a/bs [:b/id {:b/cs [:c/id]}]}] for :a resolver I declared :a/bs as output, pulled the data from datomic, and renamed the :db/id keys but in addition for whatever reason I pulled the rest of the graph but didn’t rename the keys. Pulling and returning only what I was supposed to return in the :a resolver fixed the issue. It’s still unclear to me why it sometimes worked, but doesn’t really matter.#2020-06-0708:13Mark WardleHi. Is there support in pathom for federation of resolvers between providers please? I’d like to federate or hop between resolvers running on different services - is such a thing supported or does one simply create a facade and forward on each resolution request?#2020-06-0708:14Mark WardleMy second question is one of style. I’d like to build a set of resolvers that supports different ontologies, thus abstracting implementation details and providing graph edges and graph exploration using a standardised vocabulary - this would mean clients can walk - for example, organisations across different countries/regions without concern about underlying schema differences. For example, I’m building a graph of health and care organisations in the UK and would like to represent relationships via an ontology namespace such as https://www.w3.org/ns/prov - so that a relationship ends up being https://www.w3.org/ns/prov#wasDerivedFrom - I’m planning to use reverse URL notation in the clojure namespace to represent a particular attribute but is there a better way? For instance, this means clients can fetch the name of a given organization using :org.w3.http://www.2004.02.skos.core/prefLabel I quite like the first-class nature of naming here, but can’t use traditional URL/value tuples https://www.w3.org/2004/02/skos/core#prefLabel because that wouldn’t be a valid clojure namespace.#2020-06-0813:26souenzzoThere is some EQL function that receives data + query and return a structure showting the "missing" parts?#2020-06-0817:46souenzzoDone that https://github.com/molequedeideias/eql-inspect#2020-06-0817:46souenzzoDone that https://github.com/molequedeideias/eql-inspect#2020-06-0918:19wilkerluciohello @mark354! there is some start on the federation work, but I dont feel that is good enough for general usage at this moment. so for understand, you want to kind map from RDF directly to resolvers with pathom, kinda like that?#2020-06-0919:09Mark WardleHi! Thanks. Yes - I already have a couple of health related microservices and providing a pathom facade seems like an excellent idea, but for newer services I was wondering whether I could write a pathom native server and federate rather than creating a facade. Happy to do latter however, as I guess conceptually it makes sense as the server-side partner of potentially multiple client-side applications… I’m quite keen on the ontological side of things as of course, you can see triples creating graphs fairly easily, and if you get the semantics right, you can potentially provide access to runtime clients with bags of first-class ā€˜well-known’ attributes rather than, as is usual in health, the wider model being the first-class mechanism (much like in OO).#2020-06-0919:24Mark WardlePS thank you so much for writing such a powerful library!#2020-06-0920:24wilkerluciogotcha, makes sense, your idea is right on what I like to provide for federation, distributed parsers that can be integrated. I'm a fan of RDF myself, not sure how much pathom and ontologies fit together. For the keywords I think you are doing the good approach, keywords are the closest primitive in Clojure that I can think related to properties#2020-06-0920:35Mark WardleGreat. It’s helpful to know I’m not missing something already done or completely wide of the mark! I’m hoping RDF-like triples source->property/predicate->target are a way of creating a graph by standardising the properties. I’d like to facade different types of domain information and walk graphs between and within them… at least that’s the plan! Thanks again.#2020-06-1014:59cldwalker@mark354 https://github.com/arachne-framework/aristotle and https://github.com/cognitect-labs/onto are clojure projects that have done some interesting modeling with rdf. I just saw there's an #rdf so perhaps someone there has more advice#2020-06-1019:48Mark WardleThanks. They look interesting projects - I’ll read more thank you!#2020-06-1018:51tomjkiddIs there a preferred way to handle authorization concerns with resolvers?#2020-06-1020:43wilkerlucionope, its open for you to decide, a common pattern is provide some auth-token on the environment so the resolvers can use it#2020-06-1020:55tomjkiddAlright, we have been using the environment to handle our concern, and had a false start by trying to incorporate some of the grant/role information as ::pc/input properties#2020-06-1020:56tomjkiddAnd, thank you!#2020-06-1020:56dvingoI was looking into a way to tag mutations/resolvers with auth needs, and came across the transform https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/shared-resolvers.html#connect-transform helper. I think something like this shape would get there
(defn simple-tform
  [{::pc/keys [mutate resolve] :as env}]
  (log/info "ENV is: ") (pprint env)
  (if resolve
    (assoc env ::pc/resolve
               (fn [en params]
                 (log/info "IN simple tform resolve")
                 (resolve en params)))
    (assoc env ::pc/mutate
               (fn [en params]
                 (log/info "IN simple tform mutate")
                 (log/info "env is: ")
                 (pprint (keys en))
                 (mutate en params)))))

(pc/defresolver res1 [_ _]
  {::pc/output    [::test]
   ::pc/transform simple-tform
   ::my-ns/require-auth? true
   ::my-ns/auth-roles #{:admin}}
  (log/info "Hello")
  {::test "hello this is my name"})
#2020-06-1021:01tomjkiddI like the extension to defresolver to incorporate expected roles here#2020-06-1021:01tomjkiddThank you, will continue to learn about this#2020-06-1117:00jeroenvandijkI’m trying out https://github.com/wilkerlucio/pathom-datomic and I’m not getting the results I’m expecting. I’ve set up a parser like in the documentation, but for some reason it is not able find the attribute(s) I ask for:
user=>  (time (d/pull (d/db conn) '[:flag/status] [:external/id "v145153"]))
"Elapsed time: 0.373298 msecs"
#:flag{:status #:db{:id 17592186045462}}
user=> (parser {} [{[:external/id "v145153"] '[:flag/status]}])
{[:external/id "v145153"] #:flag{:status :com.wsscode.pathom.core/not-found}}
What is a good approach to debug this?
#2020-06-1117:01jeroenvandijkI have run some commands from the tests like https://github.com/wilkerlucio/pathom-datomic/blob/master/test/com/wsscode/pathom/connect/datomic_test.clj#L508 It gives a results that confirms that Pathom can read the db#2020-06-1118:42captain.gHey there! Is there any way to set a client-side pathom parser as a remote on the client side of a fulcro 3 application? I've been trying to get it to work using pfn/pathom-remote, but that depends on fulcro.client.network, which isn't a thing in fulcro 3.#2020-06-1120:22dvingoi'm interested in this as well. There's an example in the fulcro guide code: https://github.com/fulcrologic/fulcro-developer-guide/blob/master/src/book/book/pathom.cljs#2020-06-1120:22dvingohaven't played around with it yet, but it'd be great to have a client only app that makes use of api calls via a client-side parser#2020-06-1121:27Chris O’Donnell@gekarian123 @danvingo https://github.com/codonnell/crudless-todomvc/blob/master/src/crudless_todomvc/remote.cljs is an example#2020-06-1121:30Chris O’DonnellThe client-side parser uses graphql, but the setup should be the same as with a rest api or whatever other data source.#2020-06-1121:38Chris O’DonnellAh, I should update my example to use mock-http-server. Didn't know about that!#2020-06-1202:06dvingovery cool, thank you!#2020-06-2117:38friczeHi! I’m trying to generate GraphQL from EQL using Pathom, and it works, mostly, flawless. Unfortunately I cannot see how to set operation name. Examples can be seen here https://graphql.org/learn/queries/
query HeroNameAndFriends {
  hero {
    name
    friends {
      name
    }
  }
}
query HeroNameAndFriends($episode: Episode) {
  hero(episode: $episode) {
    name
    friends {
      name
    }
  }
}
I can only query fields. No way to query operation by name. Am I missing something, is this functionality hidden? If it’s not, I’m more than happy to create PR to Pathom GraphQL generation
#2020-06-2117:38friczethanks for any help#2020-06-2118:45souenzzo@andrzej.fricze once EQL is defined in terms of arrays/map/lists and you can assoc / update these datastructures, EQL has many less features then GraphQL this query in Clojure will be something like
(defn HeroNameAndFriends
  [episode]
  `[{:HeroNameAndFriends [{(:hero {:episode ~episode})
                           [:name {:friends [:name]}]}]}])
#2020-06-2120:29friczeunfortunately this doesn’t set HeroNameAndFriends as operation name. it’s just one of query fields. maybe my assumption Was wrong from the beginning, and Pathom doesn’t intend to recreate full GraphQL query API, only a valid subset?#2020-06-2212:08souenzzo@andrzej.fricze pathom and EQL ecosystem are "another approach" GraphAPI's you can do a lot of things that graphql do, things that graphql can't do, and maybe there is things that only graphql do. pathom has the concept of placeholders, that you can use to "mimic" HeroNameAndFriends
[{:>/HeroNameAndFriends [{(:hero {:episode episode})
                          [:name {:friends [:name]}]}]}]
Also, you can create your own a function that merge args and query as GraphQL do
(letfn [(apply-args-in-query [{:keys [query args]}]
          (->> query
               eql/query->ast
               (eql/transduce-children (map (fn [{:keys [params] :as node}]
                                              (if params 
                                                (assoc node 
                                                  :params (into {}
                                                                (map (fn [[k v]]
                                                                       [k (get args v v)]))
                                                                params))
                                                node))))
               eql/ast->query))]
  (apply-args-in-query {:query '[{:>/HeroNameAndFriends [{(:hero {:episode :$episode})
                                                          [:name {:friends [:name]}]}]}]
                        :args  {:$episode 42}}))
=> [{:>/HeroNameAndFriends [({:hero [:name {:friends [:name]}]} {:episode 42})]}]
#2020-06-2216:54friczeI see, thanks so much for explanation @souenzzo#2020-06-2313:22souenzzo#2020-06-2422:36Aleedanyone use pathom’s query engine with graphql? I’ve seen Graffiti (https://github.com/denisidoro/graffiti), not sure if there’s alternative approaches. also i know pathom has an EDN->Graphql parser, in this case a Graphql->EDN parser would be useful too#2020-06-2507:08jmayaalvhi @alidcastano we are creating a prototype that uses pathom and graffiti#2020-06-2507:08jmayaalvbasically we want to use eql for all our apps, but we need to provide graphql to the outside world#2020-06-2507:09jmayaalvso we are evaluating graffiti#2020-06-2507:09jmayaalvfirst thing is that it doesn’t run on the latest version of lacinia, i created this fork to upgrade it https://github.com/jmayaalv/graffiti?organization=jmayaalv&amp;organization=jmayaalv#2020-06-2507:11jmayaalvwe are now able to create simple Graphql queries backed by pathom but we still need to test a lot of things before saying it’s good to go.#2020-06-2507:15jmayaalvi think it would be great for pathom’s reach to have an official way to offer a graphql api (@wilkerlucio has said it’s on the roadmap)#2020-06-2507:18jmayaalvby the way i am colombian too :)#2020-06-2515:04Aleedja, nice, I went back n worked remotely for 5 months there last year#2020-06-2515:17jmayaalvcool, i’m from Medellin. go there quiet often, living in Poland atm šŸ™‚#2020-06-2615:33fricze@U0J6U23FW where in Poland? šŸ˜„#2020-06-2615:34jmayaalvKrakow šŸ™‚#2020-06-2615:35friczelol šŸ˜„ man, you have to come to https://www.meetup.com/Krakow-Clojurians/ we’ve been dead last months (corona) but I’m pretty sure we’re gonna host something soon#2020-06-2615:36friczeit’s always good to meet fellow Clojurians in person#2020-06-2615:36friczeand it’s super small, super cozy meetup šŸ˜„#2020-06-2615:37jmayaalvcool, i've already registered looking forward for the next one šŸ™‚ !#2020-06-2615:38friczeawesome šŸ˜„#2020-06-2619:43souenzzoThere is a #pathom function for this?
(defn get-from-context
  [{:keys [parser] :as env} context ident]
  (let [[eid & others] context]
    (-> (parser env `[{(~eid {:pathom/context ~(into {} others)})
                       ~[ident]}])
        (get-in [eid ident]))))
#2020-06-2800:20dvingoI was working on how to organize resolver code - mainly for common concerns like auth and made an interesting connection - combining pedestal interceptors with pathom-connect transform....
(require
    [io.pedestal.interceptor.chain :as chain]
    [io.pedestal.interceptor.helpers :as ih])

(def response-key :pathom-interceptors/response)

(defn assoc-response [in-map response]
  (assoc-in in-map [:env response-key] response))

(defn get-response [in-map]
  (get-in in-map [:env response-key]))

(defn response-interceptor
  [{:keys [opts env params] :as in}]
  (let [{::pc/keys [mutate resolve]} opts
        response (get-response in)]
    (if response
      in
      (if resolve
        (assoc-response in (resolve env params))
        (assoc-response in (mutate env params))))))

(defn interceptors->pathom-transform
  "Executes vector of interceptors on a pathom resolver or mutation.
  Each interceptor is passed a single map (the environment) which has the keys:
  :opts - The definition-time pathom resolver or mutation map of options.
  :env - The pathom connect environment for the resolver or mutation passed by pathom at request-time.
  :params - The params to the resolver or the mutation.

  Responses are set on the env like so:

  (assoc-response env {:my-pathom-resp :value})
  "
  [interceptors]
  (fn pathom-transform*
    [{::pc/keys [mutate resolve] :as opts}]
    (let [interceptors (conj interceptors (ih/after response-interceptor))]
      (cond
        resolve
        (assoc opts ::pc/resolve
                    (fn [en params]
                      (let [out (chain/execute {:opts opts :env en :params params} interceptors)]
                        (get-response out))))

        mutate
        (assoc opts ::pc/mutate
                    (fn [en params]
                      (let [out (chain/execute {:opts opts :env en :params params} interceptors)]
                        (get-response out))))
        :else (throw
                (Exception.
                       (str "Attempting to use interceptor transform on a map that does not have a resolve or mutate.")))))))

(defn auth-user-interceptor
  [{:keys [opts env] :as in}]
  (let [{:auth/keys [no-user-msg]
         :or        {no-user-msg "You must be logged in to perform this action."}} opts
        {:keys [current-user]} env]
        (cond-> in (not current-user)
          (assoc-response (server-error no-user-msg)))))

(pc/defmutation create-thing-mutation
  [{:keys [current-user] :as env} props]
  {:auth/no-user-msg "You must be logged in to create a thing."
   ::pc/transform    (interceptors->pathom-transform [(ih/before auth-user-interceptor)])}
   ;; do mutation
  )
#2020-06-3020:26souenzzo(parser env [(op {:arg 1}) (op {:arg 2})]) is a antipattern?#2020-07-0101:03dvingoI think that's just due to a detail of the current fulcro server transaction processing implementation. I believe it can be updated to work with multiple mutations of the same name#2020-07-0101:20souenzzocontext: I'm working just with pathom, there is no frontend in my app šŸ™‚#2020-07-0111:55dvingoahh, yea I'm trying it out and it looks like you're right, it is happening at the pathom level:
(parser {} '[(test-mutations/test-mutation {:param 1}) (test-mutations/test-mutation {:param 2})])
=> #:test-mutations{test-mutation {}}
where the mutation just returns {} and with a different name:
(parser
    {}
    '[(test-mutations/test-mutation {:param 1})
     (test-mutations/test-mutation2 {:param 2})])

=> #:test-mutations{test-mutation {}, test-mutation2 {}} 
#2020-07-0114:14wilkerluciono anti pattern, this is a supported feature on EQL :)#2020-06-3020:27souenzzo#fulcro do not like multiple ops in transactions#2020-07-0114:14wilkerlucionot sure what you mean, fulcro is totally ok with multiple ops in transactions#2020-07-0114:15wilkerluciowhat happens is that you can't have transaction guarantees between the ops, but if they are separated operations its a fine to call as many as you want#2020-07-0117:18dvingoI think he's referring to if the operation is the same symbol#2020-07-0117:19dvingoyou get back a map so you their responses are "merged"#2020-07-0117:19dvingo[(op {:a 1}) (op {:a 2})] => {op {:answer 1}}#2020-07-0117:20dvingofor example - what if one of the calls fails, but one of them succeeds?#2020-07-0117:48souenzzoI'm using pathom-as so no problem#2020-07-0117:51dvingodo you have a link for that? I haven't heard of that#2020-07-0117:55souenzzoyou can do [(op {:pathom/as :a}) (op {:pathom/as :b})] ;;=> {:a ... :b ...} @U051V5LLP#2020-07-0117:56dvingooh interesting! thanks#2020-06-3023:22lgesslerhi, is there a way to put state into the environment on a per-query basis? i want to create a database session per query, but the body of a defresolver isn't the right place to do it because any other resolvers that are involved won't have access to it#2020-06-3023:43lgesslernvm, found the answer in the docs#2020-07-0117:02Jakub Holý (HolyJak)Hi! How do I use p/raise-errors ? Via p/post-process-parser-plugin ?#2020-07-0119:46wilkerlucioyup, that's a valid option#2020-07-0316:34Andreas EdvardssonI'm still not able to understand whether union to-many queries are allowed in EQL or not, and how to get them to work in pathom. to-one unions seems to work fine though. I have made a complete example with the resolvers and the queries. What am I missing?#2020-07-0511:32wilkerluciohello Andreas, I just tried running your example here and it just worked for me, what pathom version are you trying on?#2020-07-0511:32wilkerlucio#2020-07-0516:55Andreas Edvardsson@U066U8JQJ What..! :face_with_raised_eyebrow: Happy I got everything right then, even though it doesn't work for me... I'm on clojure 1.10.1 and pathom 2.2.30 on openjdk-13, a restart of the repl does not solve the problem. Pathom 2.2.31 didn't solve it either. Fulcro 3.2.9 is also on the classpath.#2020-07-0516:58Andreas EdvardssonHmm.. seems to work fine in 2.3.0-alpha9 :star-struck:#2020-07-0517:35Andreas EdvardssonWorks on 2.2.27 and 2.3.0-alpha9, not on 2.2.28-2.2.31#2020-07-0522:53wilkerlucioglad to hear you got it working :)#2020-07-0513:02souenzzoHow do I avoid cacheing in these cases?
(let [register [(pc/resolver
                  `a {::pc/params [:v]
                      ::pc/output [:a]}
                  (fn [env _]
                    {:a (p/params env)}))]
      parser (ps/connect-serial-parser register)]
  (parser {} `[(:a {:v 1})
               (:a {:v         2
                    :pathom/as :b})]))
#2020-07-0614:36Eric Ihlihttps://github.com/fulcrologic/fulcro-template/search?q=ring%2Frequest&amp;unscoped_q=ring%2Frequest How does mutation-env get a ring/request key in the lines of that code? I'm printing (keys env) in a defmutation that I have and it doesn't show up. I am searching through ring middleware and fulcro/pathom examples and I haven't yet figured it out.#2020-07-0614:40Eric IhliAh. Never mind. Just found it. https://github.com/fulcrologic/fulcro-template/blob/b813b94ce191c8d04ccceabd8d39b88b31c59cf5/src/main/app/server_components/middleware.clj#L23#2020-07-0801:36cjmurphyIs there documentation on how to specify a mutation that requires particular input/s? My use case is a mutation that needs to know the current user (which is available via an output only current-user-resolver).#2020-07-0802:18souenzzo@cjmurphy I just do (let [user-data (parser env [...])]) inside mutation But I think that you can do something with pc/transform too#2020-07-0802:19souenzzoAlso I'm trying to develop a "style guide" to help us to know/remember/develop that kind of "pattern" in EQL ecosystem :) https://github.com/souenzzo/eql-style-guide#pathom#2020-07-0806:11cjmurphyRe. your style guide there is a comment that doesn't seem to apply to the code below it.#2020-07-0806:16cjmurphyI called the parser again, but from a function so can explicitly see all the cases. As it happens the current user is in the env so I could have just grabbed directly.#2020-07-0806:20cjmurphyWhat would make sense to me would be if inputs were 'first class citizens' for mutations, which I think can be done with a plugin. And I was just wondering if it had been documented. Possibly that's the pc/transform you mentioned. For me is seems nice to only use standard plugins.#2020-07-0813:05souenzzoSomething like this? https://gist.github.com/souenzzo/362819e5088db5f4a089103602dc68f0#2020-07-0820:44cjmurphyCurrently every mutation has 2 args, first being env and second being params. If inputs were first class citizens then there would be 3 args, with the extra one being inputs. I guess it is some kind of a plugin that would provide that. Not sure your code is that generic. I know @U066U8JQJ was experimenting with something that did that. Not sure if it was finished or documented.#2020-07-0820:54wilkerlucio@cjmurphy they are different concerns, params are for mutations what input is for resolvers, but they are different in the sense that params are sent as-is, you can make a plugin to auto-resolve the params on mutations, the arity 3 could be a thing, but not sure at this moment#2020-07-0818:03Eric IhliI see this mutation in an example. Does pathom guarantee mutations are run in order? Does it depend on configuration (https://wilkerlucio.github.io/pathom/#_parallel_parser)? [(log-in ~credentials) (finish-log-in {})])#2020-07-0819:23souenzzo@ericihli [(op-1 {}) (op-2 {})] ~> both MAY be executed at same time (parallel parser will, serial parser don't [{(op-1 {}) [(op-2 {})]}] -> op-2 will only start after op-1 finish. But in general, isn't a good idea compose mutations in EQL Prefer [(log-in+finish-log-in ~creds)]#2020-07-1018:26pithylessJust watched your Clojure/North talk @wilkerlucio. Good stuff! It's nice to hear someone confirming that the ideas I've been working with (wrapping core libraries, env flow) are not crazy and others are doing it as well. The one thing I constantly go back and forth on is the naming conventions for keywords. Based on your pedestal example:
{:io.pedestal/request-headers ...
 :io.pedestal/request-body ...
 :io.pedestal/response-status ...}

vs

{:io.pedestal.request/headers ...
 :io.pedestal.request/body ...
 :io.pedestal.response/status ...}
Have you got any insights or wisdom to share about keyword namespacing conventions? Lingering questions: • Do you prefer application or domain entity nses? • Should namespaces be real nses (so you can ::prefix/* ) or should we not care? • What about storing to external dbs with limited semantics? (Are you using edn-json everywhere?) • What about datomic enum conventions? (i.e. :some.entity.category/enum )
#2020-07-1021:36wilkerluciohello @U05476190, glad you enjoyed the talk šŸ™‚ I think this is a question about how to model the system, and there is no one answer. I'm not 100% sure either, and takes trial and error until you get confidence. What I can tell you from my experience is that I try to keep things flat, IME flat is simpler, and when makes sense, I do it. In the pedestal example, when I modelled that I considered the HTTP as the domain boundary of that namespace, and I only though hard enough for a demo in a presentation, altough I played with both examples you sent when modeling fetch API, and I like the flat (first, less namespaces) better. Another important thing to take in account, I personally miss Clojure having the feature for lightweight aliases, there is a pending issue on Jira for that: https://clojure.atlassian.net/browse/CLJ-2123 I believe this feature would make easy enough so we were set free to make more namespaces, than my opinion may change, but right now doing the flat is just more convenient#2020-07-1021:38wilkerlucioabou the lingering questions: Do you prefer application or domain entity nses? - I dont see a difference, the application must have a domain, and then model around the domain Should namespaces be real nses (so you canĀ ::prefix/*Ā ) or should we not care? - they dont need to be, sometimes I create then for convenience, we can hope for https://clojure.atlassian.net/browse/CLJ-2123 What about datomic enum conventions? (i.e.Ā :some.entity.category/enumĀ ) - I like that, and again CLJ-2123 could make then accessible#2020-07-1418:10souenzzoComment any node of this query, and it will work but these nodes causes a exception
(let [parser (ps/connect-serial-parser
               {::ps/connect-reader pc/reader3}
               [(pc/constantly-resolver :f "f")
                (pc/constantly-resolver :e "e")])]
  (parser {}
          '[{:>/a [(:e {:p :p1})]}
            {:>/b [(:e {:p :p2})]}
            {:>/c [:f]}
            {:>/d [:f]}]))
#2020-07-1418:28souenzzo@U066U8JQJ should I open a issue?#2020-07-1419:04souenzzo
(p/lift-placeholders-ast {}
                         {:type     :root
                          :children [{:type         :join
                                      :dispatch-key :>/a
                                      :key          :>/a
                                      :children     [{:type :prop :dispatch-key :e :key :e}]}
                                     {:type         :join
                                      :dispatch-key :>/b
                                      :key          :>/b
                                      :children     [{:type :prop :dispatch-key :e :key :e :params {}}]}
                                     {:type         :join
                                      :dispatch-key :>/c
                                      :key          :>/c
                                      :children     [{:type :prop :dispatch-key :f :key :f}]}
                                     {:type         :join
                                      :dispatch-key :>/d
                                      :key          :>/d
                                      :children     [{:type :prop :dispatch-key :f :key :f}]}]})
#2020-07-1419:25souenzzoI think that it's the root
(p/merge-queries 
  [:a]
  '[(:a {})])
#2020-07-1505:01wilkerlucioyes, open the issue please šŸ™#2020-07-1512:15souenzzoI end up seen that the issue is: When you try to merge to nodes with different params, the merge returns nil lift-placeholders ignores the "nil" and continue working and create wired results#2020-07-1512:17souenzzoi think that maybe merge-queries should throw something like can't merge nodes a and b But sure, it's a breaking change, so pc/reader4#2020-07-1513:28wilkerlucionah, reader3 is mostly an experiment, and tbh I think the merge-queries shouldnt be like that, Im considering making the planner aware of placeholders so he can deal with it directly, this may be a better way to avoid those#2020-07-1617:47daniel.spanielanyone know any tricks for doing pagination with datomic with pathom ( would like to do filtering and sorting too ) if i have to do in memory i can .. but its painful#2020-07-1619:57wilkerluciothe problem here is how to paginate datomic, they dont have a good solution, I spent a good time trying to get that to be efficient, but no good, the best I could do was paginate by date ranges, this way you can constrain space and make it usable (but has the trade-offs from date pagination, for example, the number of items for different date ranges may change dramatically)#2020-07-1715:33daniel.spanielthanks wilker .. will ponder some more then ..#2020-07-1723:24Joe R. SmithIf you're running your code on Datomic Cloud Ions or a Peer server doing the sort/pagination in memory is pretty close to order by/limit in SQL- it's happening in the database process. Consider using range queries, where possible. There's also a new index-pull API you should check out: https://docs.datomic.com/cloud/query/query-index-pull.html#2020-07-2400:21daniel.spanielthanks @U087E7EJW good tips#2020-07-1723:26Joe R. SmithI'm playing with pathom-datomic using Datomic Cloud and getting class clojure.lang.ExceptionInfo: No output available - #:com.wsscode.pathom.connect{:sym com.wsscode.pathom.connect.datomic/datomic-resolver whenever I query for a property. My Parser is pretty simple:
{::p/env     {::p/reader [p/map-reader
                            pc/reader2
                            pc/open-ident-reader
                            p/env-placeholder-reader]
                ::p/placeholder-prefixes #{">"}}
   ::p/mutate  pc/mutate
   ::p/plugins [(pc/connect-plugin {::pc/register resolvers})
                (pcd/datomic-connect-plugin (assoc client-config
                                              ::pcd/conn conn
                                              ::pcd/whitelist #{:db/id :user/id :user/email}))
                p/error-handler-plugin
                p/trace-plugin]}
A query like this returns that error:
(parser {} [{[:user/email "
#2020-07-1723:49Joe R. SmithUPDATE: Looks like the fix was to use pc/reader3#2020-07-1800:11Joe R. SmithI'm finding a few bugs as I try to use this library-- @wilkerlucio would you prefer issues for the bugs or PRs? šŸ™‚#2020-07-1914:58wilkerluciohello, awesome! if you think its an obvious bug, feel free to send PR, otherwise you can open a issue for discussion#2020-07-1914:59wilkerlucioplease note I consider reader3 is more a beta thing, given that it wasn't battle tested in any case as far as I know (while reader2 is heavily used on production)#2020-07-1917:29Joe R. SmithShould reader2 work? I get errors (you have to run it again to see what they are) when using reader2.#2020-07-2005:54wilkerluciosorry, in this case no, datomic plugin requires reader3, I wrote that but never used myself, so consider experimental as this stage (on top of the reader3 stage as well). those things needs more maturity and testing, I wrote those as to try ideas but never got to try then for real, so I warn to use those with that in mind, and just curious, are you currently just playing with it or using for some project?#2020-07-1800:11Joe R. Smithor both, I guess.#2020-07-1800:11Joe R. Smithreally cool library, btw.#2020-07-1914:59wilkerluciothanks!#2020-07-2121:32tvaughanWhat's the canonical way to query for enums in EQL/Pathom? If I have a schema like:
{:db/ident :document/status
  :db/valueType :db.type/ref
  :db/cardinality :db.cardinality/one}
 {:db/ident :document.status/draft}
 {:db/ident :document.status/final}
A query for [:document/status] returns :document/status {:db/id 17592186045468}. I know that with Pathom it's recommend to have separate id's for entities, like :document/id, :person/id, etc. which I do have elsewhere. But entities like these aren't enumerated entities. If there's an EQL compatible way to query for the keyword, I haven't found it. Any help is appreciated. Thanks
#2020-07-2121:38eoliphantthis isn’t really a ā€˜pure’ pathom thing per-se. Just has to do with the fact that while EQL/Pathom are very close, there isn’t always 1-to-1 correspondence with Datomic. so you can either roll your own resolvers for your enum attrs that return the :db/ident value, or check out pathom-datomic that has some special sauce to make this easier. basically you give it your enum attributes at config time, and it does what I described for you#2020-07-2121:43tvaughanRight, I knew I was conflating the two. Sorry for any confusion. Thanks @eoliphant. I'll check-out pathom-datomic#2020-07-2121:46eoliphantah no, not at all. if you can use Datomic in your app, it with Pathom is a match made in heaven lol. Just little stuff like that you have to be aware of. And things just got a bit better with some of the new datomic apis that were just released. qseq and index-pull#2020-07-2121:48tvaughanCool. Thanks again!#2020-07-2214:04souenzzoAlso @tvaughan as a frontend/fulcro dev, it's useful to have :document/status {:db/ident :document.status/draft} because it opens space to things like :db/ident :document... ::label "A draft" ::description "You can't sent it"#2020-07-2214:11souenzzoPS: You can save in datomic
:db/ident :document.status/draft
:app.enum/description "abc"
:app.enum/label            "abc"
PSPS: No IDK how to do i18n šŸ˜ž
#2020-07-2214:52tvaughanSorry @U2J4FRT2T, I don't follow. I understand how these things work in the Datomic pull-syntax world, but it seems this part at least isn't compatible with EQL/Pathom. What is this meant to show? Sorry, I'm not being cute. I don't understand this#2020-07-2214:57souenzzoWhat I'm saying is that in practice, ask for [{:document/status [:db/ident]}] is a good thing It's wired at first look, but once you get into a #fulcro app, you will see that it opens space to request "metadata" about your enum, like it's label#2020-07-2214:58souenzzoIn my PS, i tryied to show that datomic allow you to store metadata in the same entity that you store your enum#2020-07-2215:00tvaughanGotcha. That's an interesting approach. Thanks!#2020-07-2215:02souenzzoAlso: https://github.com/souenzzo/eql-style-guide/issues/5#2020-07-2220:44jacksonWhen using pathom with datomic, what’s the recommended way to add a db to the environment for resolvers? I’m currently using the env-wrap-plugin to add the connection to the environment, but I’d like for each resolver to use the same db. Of course I could add the db rather than the connection in env-wrap-plugin, but is there somewhere else to add the connection just once rather than connecting every time? I’ve tried adding keys to the parser ::p/env but that doesn’t seem to be what’s passed to env-wrap-plugin.#2020-07-2220:49jacksonI suppose I could just create the connection elsewhere in my ns and just add it with the env-wrap-plugin for use with mutations.#2020-07-2407:12Quentin Le GuennecDoes pathom integrate well with re-frame?#2020-07-2412:21souenzzo@quentin.leguennec1 I don't think that there is a re-frame<>EQL integration library But you can use a EQL backend(with pathom) as you use REST or GraphQL backend. Something like this
:http-xhrio {:method          :post
             :uri             "/api"
             :params          `[{:my/current [:query
                                              :with
                                              :params]}]
             :response-format (ajax/transit-response-format)
             :on-success      [:good-http-result]
             :on-failure      [:bad-http-result]}
#2020-07-2412:26Quentin Le Guennec@souenzzo I see, thank you#2020-07-2520:36lgesslerhi, I'm trying to implement some simple authentication via a pathom transform and i'm having some trouble when I want to return something that's not a result of calling resolve. putting code in reply#2020-07-2520:37lgesslerso I define my transform:
(defn auth-tx [{::pc/keys [resolve] :keys [auth] :as outer-env}]
  "Transform for a Pathom resolver that checks whether the user has sufficient permissions
  for a given operation. Checks the resolver's env for the :auth keyword, which can
  currently be either :user or :admin"
  (assoc
    outer-env
    ::pc/resolve
    (fn [env params]
      (if (authorized env auth)
        (resolve env params)
        {:server/error? true :server/message "..."}))))))
#2020-07-2520:37lgesslerand then I put it on my resolver:
(pc/defresolver all-users-resolver [{:keys [neo4j]} _]
  {::pc/output    [{:all-users [:user/id]}]
   ::pc/transform mc/auth-tx
   :auth          :admin}
  ;;...
  )
#2020-07-2520:38lgesslerwhen the user is authorized to perform the action, everything works OK, but if they're not, I get an error: Exception in thread "async-dispatch-15" java.lang.IllegalArgumentException: No implementation of method: :take! of protocol: #'clojure.core.async.impl.protocols/ReadPort found for class: clojure.lang.PersistentArrayMap#2020-07-2520:39lgesslerI'm confused because as far as I can tell (resolve env params) is just returning a plain map--when I replace it with {:all-users []} I don't get this :take! error. so shouldn't it be OK for me to return that map literal in the non-authorized case? is there some kind of function i need to use to wrap it?#2020-07-2613:15wilkerlucioyes, looks strange, what parser and reader are you using?#2020-07-2615:14lgesslerp/parser and pc/parallel-reader: https://github.com/lgessler/glam/blob/master/src/main/glam/server/pathom_parser.clj#L46#2020-07-2616:03lgessleri'll try the sync reader in a sec and see what happens#2020-07-2616:20lgesslerconfirmed that it doesn't crash with pc/reader , though the return value is [{:all-users {}}] instead of {:server/error? true :server/message "..."} for some reason... anyway, it seems like it succeeds in preventing the call to the resolver to run, so this is good enough šŸ™‚ the error message getting to the client isn't really important anyway#2020-07-2616:21lgesslerwould still be curious to know what i was doing wrong though... and i don't understand how big the perf difference is between parallel-reader and reader but i hope it's not terribly great#2020-07-2616:21lgesslerthanks for the help @U066U8JQJ!#2020-07-2616:22wilkerlucioyeah, with the parser you should use the reader2, parallel-reader is only compatible with parallel-parser, this should fix the issue :)#2020-07-2616:23wilkerluciosorry the messy confusion around parsers and readers, thats I pain point I plan to fix #2020-07-2616:28lgesslerok great, thanks! also wow i just realized i had both parallel-reader and reader2 in my ::p/reader... i'm surprised that worked at all šŸ˜…#2020-07-2617:22wilkerluciothey work in a chain processing, so a reader can operate or delegate to the next, if you have then, what would happen is that the first would pick up#2020-07-2709:10ak-coramone of my resolvers always gets called twice with the same input, are there any common mistakes that can lead to this?#2020-07-2713:12yendait's normal if you are doing a parallel parser and some fields are resolved in parallel withthe same input. For instance if you are resolving a list of comment and the user associated with each comment while the user commented more than once#2020-07-2713:11yendais there a way to not return keys instead of pathom-not-found? or at least nil so that booleans don't end up being truthy?#2020-07-2713:39souenzzo@yenda there is a p/elide-special-outputs-plugin plugin šŸ˜‰#2020-07-2713:41yendawhat does it do exactly?
#2020-07-2713:47souenzzoSomething like it
(letfn [(elide-special
          [el]
          (cond
            (map? el) (into {}
                            (keep (fn [[k v]]
                                    (when-not (contains? #{::p/not-found
                                                           ::p/error}
                                                         v)
                                      [k (elide-special v)])))
                            el)
            (coll? el) (mapv elide-special el)
            :else el))]
  (let [result (<! (parser env tx))]
    (elide-special result)))
#2020-07-2813:50jacksonIs it possible to update the environment from a mutation for a subsequent mutation join?#2020-07-2815:38wilkerlucio@jackson.reynolds yes, here is an example:
(ns com.wsscode.pathom.union-repro
  (:require
    [clojure.core.async :as async]
    [com.wsscode.pathom.core :as p]
    [com.wsscode.pathom.connect :as pc]))

(pc/defresolver x [env {:keys []}]
  {::pc/output [::x]
   ::pc/cache? false}
  {::x (:x env)})

(pc/defmutation sample-call [env _]
  {}
  {:foo    "bar"
   ::p/env (assoc env :x "meh")})

(def resolver-registry [sample-call x])

(def parser
  (p/parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/reader2
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate
     ::p/plugins [(pc/connect-plugin {::pc/register resolver-registry})
                  p/error-handler-plugin
                  p/trace-plugin]}))

(comment
  (parser {:x "bla"}
    [::x
     {(list `sample-call {}) [::x]}])
  
  ; => {::x         "bla"
  ;     sample-call {::x "meh"}}
  )
#2020-07-2816:43jacksonThanks. That was my first inclination as well but it seems that as soon as I add ::p/env to the return map I get a NullPointerException. I can prn the new environment just fine and I’m not asking to see it in the EQL query. If I add the environment as ::env instead, I don’t get an exception and obviously don’t update the environment; but as soon as I change it to ::p/env I get the exception.#2020-07-2816:57jacksonIn fact, I still get {:com.wsscode.pathom.core/reader-error ā€œclass java.lang.NullPointerExceptionā€} with your example.#2020-07-2817:07jacksonThe only main difference I’m seeing is I’m using p/parallel-parser with pc/parallel-reader and pc/mutate-async#2020-07-2817:30jacksonFigured it out. Updated from 2.2.20 to 2.2.30 and everything works as expected. Whew#2020-07-3012:16Thijs CreemersI am trying to use pathom to send graphql queries to an exsting backend, In the documentation there are a few examples, but when I am trying the examples it appears as if I am working with an older library API. My Fulcro version is 3.0.10 and the pathom version is 2.2.31. Has anyone a hint?#2020-07-3014:03wilkerluciohello, and yes, those docs are still on Fulcro 2, Pathom provides no direct helpers for Fulcro 3 at this point, you need to create a custom Remote on Fulcro 3, and hook the parser into it#2020-07-3014:04wilkerluciohere is an example of writing a remote like this for Fulcro 3 https://gist.github.com/wilkerlucio/c3c76024da0807223592cdd5bcd71d0e#2020-07-3014:04Thijs CreemersThanks for the reply#2020-07-3014:18souenzzoI'm using it here: https://github.com/souenzzo/eql-realworld-example-app/blob/master/src/conduit/client.cljs#L20#2020-07-3014:19Thijs CreemersThat will be helpful, thanks#2020-07-3014:04wilkerlucio#2020-07-3015:26joshkhhello! is it possible to alias mutations? unfortunately i'm not getting the results i expected, but maybe my syntax is incorrect:
(pg/query->graphql '[{(myMutationFn {:com.wsscode.pathom.graphql/alias "alias1" :id "someid" :user "someuser"}) [:id :user]}
                     {(myMutationFn {:com.wsscode.pathom.graphql/alias "alias2" :id "someid" :user "someuser"}) [:id :user]}])
=>

; missing aliases:

"mutation {
   myMutationFn(id: \"someid\", user: \"someuser\") {
     id
     user
   }
   myMutationFn(id: \"someid\", user: \"someuser\") {
     id
     user
   }
 }
 "
#2020-07-3113:07ziltiWhat are the reasons that sometimes resolvers are called with nil as param despite there being a key specified that is actually there?#2020-07-3116:00wilkerluciocan you make a small demo case?#2020-07-3117:08ziltiI'll try to. This happens inside Fulcro though and I don't know what exactly is going on behind the scenes there#2020-07-3114:57yendaI am using pathom standalone within a re-frame app, I am reading though fulcro doc because I'd like to improve the way I do data normalization. Currently I parse the server response with defmethods to normalize it into app-db. I am under the impression that fulcro is doing normalization under the hood, does it manage to get normalized response from the pathom server? Is it possible to get that without fulcro? Not getting a deeply nested tree from the pathom parser but a flatter structure instead? for instant in a big query that ask for a list of user items, get the users details at the same level as the items details, so that when multiple items belong to a single user, the user and his details are only present once in the response, while the item only contains the user id#2020-07-3116:02wilkerlucionormalization is done by Fulcro, pathom plays no part in it, Fulcro does this by annotating each query fragment (on meta data) with the component that requested it, then fulcro walks the query, computing the identity of each part of the response (this is what the ident thing is about)#2020-07-3116:33yendaSo there is no normalization server side?#2020-07-3116:39Chris O’DonnellThe response is not normalized on the server, no.#2020-07-3116:44Chris O’DonnellThat said, nothing is stopping you from implementing an endpoint which normalizes pathom results before sending them to the client. No doubt there would be pitfalls, but I'm not sure offhand what they might be.#2020-08-0212:50yendaI'm trying pathom-viz with the connector and I don't get anything, just
WARN [taoensso.sente:1287] - Chsk is closed: will try reconnect attempt (110) in 1000 ms
Waiting for channel to be ready 1000
Waiting for channel to be ready 1000
over and over in my logs. What am I doing wrong here?
#2020-08-0321:00lgesslerIs there some config I need to set up for a defmutation for its output? If my mutation errors, I want to return a map with the keys :server/error and :server/message but by the time pathom's response gets to ring I see that the map is empty. do keys in mutation return values get trimmed somehow?#2020-08-0321:04lgesslerOK, so I was triggering the mutation using uism/trigger-remote-mutation with the ::m/returning option set--getting rid of ::m/returning resolved this issue for me. still surprised this happened, though, since as far as i can tell using the ::m/returning option didn't change the tx being sent over the wire to pathom#2020-08-0415:10wilkerluciohello, when you use returning Fulcro does change the query it sends to Pathom, it adds a join on the mutation call, so you get the data, I believe the problem you are having is that, because you have a join query on the mutation, that join query may not have the ::server/error keys, so pathom will strip those out of the response (because the request didn't mentioned it)#2020-08-0415:10wilkerluciothe simplest fix is to add those keys to the component you are using the returning with#2020-08-0415:11wilkerlucioor add a config to pathom to always spit out those keys using mutation join globals: https://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/connect-mutations.html#_mutation_join_globals#2020-08-0415:27lgesslermakes sense, thank you!#2020-08-0418:22Tyler NisonoffHey all - beginner question I'm going through the book, and cannot get the parameters section to work I found https://github.com/wilkerlucio/pathom/blob/3242250e03f5f6b4c5faae1162e6358e9c9a36c1/docs-src/modules/ROOT/examples/com/wsscode/pathom/book/connect/parameters.cljs But I always get:
Execution error (ExceptionInfo) at com.wsscode.pathom.parser/expr->ast (parser.cljc:104).
Invalid expression 
Running the parser as specified in the comment block I'm using pathom version 2.2.31, clojure 1.10, etc. -- am I missing something obvious?
#2020-08-0418:46souenzzo@tylernisonoff Where is (<!! (parser {} [(::instruments {:sort :instrument/brand})])) Should be (<!! (parser {} '[(::instruments {:sort :instrument/brand})]))#2020-08-0418:48souenzzoWhen you do [(:ident {:param 42})] on Clojure/REPL it actually evaluates to [nil], which is a invalid EQL On Clojure/REPL, when you need "a EDN list", like (1 2 3) you need to quote it.#2020-08-0418:50Tyler Nisonoffahhh got it, duh šŸ™‚ Thank you! Think it's worth me submitting a PR to update this file in the examples?#2020-08-0418:54souenzzoPR are welcome šŸ™‚ cc @U066U8JQJ#2020-08-0418:54Tyler Nisonoffcool will do!#2020-08-0420:56Tyler Nisonoffhttps://github.com/wilkerlucio/pathom/pull/169 I included a note in the docs about this in the PR, but can remove that if it's unwarranted. (or happy to re-word it)#2020-08-0512:27souenzzoadded a small comment#2020-08-0512:51Tyler Nisonoff:face_palm: thanks for catching -- updated#2020-08-0512:55Tyler NisonoffThe "re-request review" button doesn't seem to do anything for me.. :thinking_face: :man-shrugging:#2020-08-1900:31Tyler Nisonoff@U2J4FRT2T when you can, mind reviewing the PR again?#2020-08-1900:32souenzzoJust @U066U8JQJ can merge#2020-08-0812:31yendaIs is possible to make the parser use idents in nested queries and put the node at the top level of the output to avoid repetition? Example:
{[user/id "x"] {:user/name "bob}
 :comments [{:message "yo" :>/user [:user/id "x"]}
            {:message "hi" :>/user [:user/id "x"]}]}
#2020-08-0813:39wilkerluciofrom pathom perspective, no, ident normalization is not a concept in pathom, given in Pathom idents just mean setting a value. this also respects a principle that the shape of the response must match the shape of the query, this avoids the user having to deal with complicated things to understand the output shape, by always being the same is simple and predictable#2020-08-0813:39wilkerluciowhat you can do is write some post-processing plugin to move things around after the response is built, makes sense?#2020-08-0814:24yendais the placeholder plugin a post processing too? Because what I want to do is basically instead of putting the content, put the indent and move the content to the top level#2020-08-0816:22wilkerlucioI don't fully understand yet what you are trying to do. can you send a more complete example, with the queyr, and an example of the result structure you are trying to achieve?#2020-08-0812:32yendaThe motivation for that being obviously that it can drastically reduce the size of the response, and it also fits much better with libraries like re-frame which will be perfectly capable of exploiting normalized data like this, instead of having to re-parse and normalize client side#2020-08-0917:01andrewzhurovHeyo, fellas! šŸ‘‹#2020-08-0917:03andrewzhurovI'm new to the tech, been trying to run locally
[{([:customer/id 123] {:pathom/context {:customer/first-name "Foo" :customer/last-name "Bar"}})
  [:customer/full-name]}]
and it fails with
1. Unhandled java.lang.IllegalArgumentException
   Key must be integer

    APersistentVector.java:  294  clojure.lang.APersistentVector/invoke
                      REPL:   96  app.server-components.pathom/eval48368
                      REPL:   95  app.server-components.pathom/eval48368
             Compiler.java: 7177  clojure.lang.Compiler/eval
             Compiler.java: 7132  clojure.lang.Compiler/eval
                  core.clj: 3214  clojure.core/eval
                  core.clj: 3210  clojure.core/eval
    interruptible_eval.clj:   91  nrepl.middleware.interruptible-eval/evaluate/fn
                  main.clj:  437  clojure.main/repl/read-eval-print/fn
                  main.clj:  437  clojure.main/repl/read-eval-print
                  main.clj:  458  clojure.main/repl/fn
                  main.clj:  458  clojure.main/repl
                  main.clj:  368  clojure.main/repl
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  155  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  190  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  189  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java:  830  java.lang.Thread/run
#2020-08-0917:04andrewzhurovBit more of context:
(defn build-parser [db-connection]
  (let [real-parser (p/parallel-parser
                      {::p/mutate  pc/mutate-async
                       ::p/env     {::p/reader               [p/map-reader pc/parallel-reader
                                                              pc/open-ident-reader p/env-placeholder-reader]
                                    ::p/placeholder-prefixes #{">"}}
                       ::p/plugins [(pc/connect-plugin {::pc/register all-resolvers})
                                    (p/env-wrap-plugin (fn [env]
                                                         ;; Here is where you can dynamically add things to the resolver/mutation
                                                         ;; environment, like the server config, database connections, etc.
                                                         (assoc env
                                                                :db (d/db db-connection)
                                                                :conn db-connection
                                                                :config config)))
                                    (preprocess-parser-plugin log-requests)
                                    p/error-handler-plugin
                                    p/request-cache-plugin
                                    (p/post-process-parser-plugin p/elide-not-found)
                                    p/trace-plugin]})
        ;; NOTE: Add -Dtrace to the server JVM to enable Fulcro Inspect query performance traces to the network tab.
        ;; Understand that this makes the network responses much larger and should not be used in production.
        trace?      (not (nil? (System/getProperty "trace")))]
    (fn wrapped-parser [env tx]
      (async/<!! (real-parser env (if trace?
                                    (conj tx :com.wsscode.pathom/trace)
                                    tx))))))

((build-parser (client/get-conn))
 {} [{([:customer/id 123] {:pathom/context {:customer/first-name "Foo" :customer/last-name "Bar"}})
      [:customer/full-name]}])
com.wsscode/pathom {:mvn/version "2.2.31"}
#2020-08-0917:04andrewzhurovAny leads?#2020-08-0917:05andrewzhurovSorry if it had been asked, googled something close in clojurians slack log, but the site appears down for me#2020-08-0917:36jaihindhreddyThe "Bit more of context" part isn't necessary. Clojure vectors can be called as functions. You can call vectors with indices, and get back elements at those indices. For example, ([:a :b] 0) evaluates to :a and ([:a :b] 1) evaluates to :b. In your case, you're trying to call [:customer/id 123] like a function, but with something that's not an int (like indices). So, you're getting the IllegalArgumentException#2020-08-0917:37jaihindhreddyYou can quote the list to get around this. For example, try ([:a :b] 100) and '([:a :b] 100) in your REPL.#2020-08-0919:02andrewzhurov> Clojure vectors can be called as functions being my third year in production I learn that only now haha šŸ˜… thank ya#2020-08-1017:47Tyler Nisonoffcan a pathom mutation utilize resolvers to load data, or can it only utilize parameters explicitly passed to it? for example, if I pass a user/id to the mutation, can i use my already written resolves to load other fields, or would i have to load them from explicitly from whatever data-source I’m using?#2020-08-1017:52souenzzo@tylernisonoff you can do
(pc/defmutation do-thing [{:keys [parser]
                           :as   env} {:user/keys [id]}]
  {::pc/params [:user/id]}
  (let [eid [:user/id id]
        user-with-address (-> env
                              (parser [{eid [:user/id
                                             :user/address]}])
                              (get eid))]
    ...))
Yeah, I agree with you that pc could implement something like ::pc/resolved-params that take ::pc/params as input and resolve it.
#2020-08-1212:43souenzzoUsing ::pc/transform
(let [connect-params (fn [{::pc/keys [mutate]
                           ::keys    [connected-params]
                           :as       mutation}]
                       (if connected-params
                         (assoc mutation
                           ::pc/mutate (fn [{:keys [parser] :as env}
                                            [eid & {:as others}]]
                                         (let [result (parser env `[{(~eid ~{:pathom/context others})
                                                                     ~connected-params}])]
                                           (mutate env (get result eid)))))
                         mutation))
      parser (ps/connect-serial-parser
               [(pc/mutation
                  `op {::pc/params        [:a]
                       ::connected-params [:a :b]
                       ::pc/transform     connect-params}
                  (fn [_ params] params))
                (pc/resolver
                  `b {::pc/input #{:a} ::pc/output [:b]}
                  (fn [_ {:keys [a]}]
                    {:b (str [:b a])}))])]
  (parser {} `[(op {:a 42})]))
#2020-08-1112:37andrewzhurov> TheĀ `::pc/params`Ā is currently a non-op In mutations, to ensure only the desired data is being transacted, is there a nice way to select a shape from params ? shape described with clojure.spec or EQL maybe#2020-08-1401:10lilactownhave a pathom/eql question: is it possible to have a disjoint union in a query? E.g. I have a folder tree where you can either have files, or more folders.
{::folder/id "x"
 ::folder/name "downloads"
 ::folder/children [{::file/id "y"} {::folder/id "z"}]}
what I’d like to do is have a query like:
[{::folder/all [::folder/id
                ::folder/name
                {::folder/children [::folder/node
                                    ::file/node]}]}]
#2020-08-1401:11lilactownwhere ::folder/node & ::file/node are resolvers that will look up the file / folder by ID#2020-08-1401:12lilactownthe above works, but I get lots of :com.wsscode.pathom.core/not-found values which aren’t very useful. hence the desire for a disjoint union, which would return either one or the other#2020-08-1401:29souenzzo@lilactown you can use p/elide-special-outputs-plugin to remove ::p/not-found Also you can use unions
(let [register [(pc/constantly-resolver
                  :folder/all
                  [{:folder/id       1
                    :folder/name     "a"
                    :folder/children [{:folder/node "folder"}
                                      {:file/node "file"}]}])]

      parser (p/parser {::p/plugins [(pc/connect-plugin {::pc/register register})]})
      env {::p/reader               [p/map-reader
                                     pc/reader2
                                     pc/open-ident-reader
                                     p/env-placeholder-reader]
           ::p/placeholder-prefixes #{">"}}]
  (parser env [{:folder/all [:folder/id
                             :folder/name
                             {:folder/children {:folder/node [:folder/node]
                                                :file/node [:file/node]}}]}]))
Personally, i don't like unions.
#2020-08-1401:30souenzzohttps://github.com/edn-query-language/eql#unions#2020-08-1401:36wilkerlucio@lilactown using the elide to remove the output is probably the best in your case, unions add more value when the queries are very distinct (depending on some "type" notion that the union will use to select the branch), this way the union can avoid a lot of unescessary processing by running the query for only that type, but for smaller cases you can just ask all and use what is returned (looks more like your case)#2020-08-1403:00lilactownThanks, I’ll try that tomorrow#2020-08-1403:00lilactownI suppose the other way of handling it would be to have separate keys where I keep folder refs and file refs?#2020-08-1414:28uwoRemind me, is it considered an anti-pattern to create a resolver that just takes the ident and eql and passes it to datomic.api/pull? I can't remember where I thought I saw Wilker say that it was better to do each attribute individually.#2020-08-1415:05wilkerlucionot a problem, but you may ask datomic to process more things than it knows about, I would check if that affects performance in any way, also remember that idents, params and unions are not valid syntax on datomic pull, so sending those there might break things#2020-08-1415:07wilkerlucioideally you should ā€œtrimā€ the query to send only what datomic can respond, which is what pathom-datomic does, but thar uses apis from pathom that are not recommended for general usage yet (reader3)#2020-08-1415:13souenzzoI do that I use https://github.com/souenzzo/eql-datomic/ But there is some wired behaviors due cache like this:
(let [register (pc/resolver
                 `a+b
                 {::pc/output [:a :b]}
                 (fn [{:keys [ast]} _]
                   (d/pull db (eql.d/ast->query ast) eid)))]
  (parser env [{:>/a [:a]}
               {:>/b [:b]}]))
parser will enter :>/a, it need [:a] call a+b it will return just {:a 42} Then parser will enter :>/b, it need [:b]. It already called a+b, so it will take the result from cache and you will not get :b This is the behavior of reader2 reader3 has another behavior, it may not happen.
#2020-08-1414:43jeroenvandijk@uwo I’m guessing using datomic.api/entity directly is more efficient, because datomic.api/pull returns a normal map whereas (:some-attribute (datomic.api/entity db id)) potentially returns another entity that can used in subqueries. But just guessing here#2020-08-1520:27Frank HenardHello, I'm trying to hook up mutations on the server side, and it's not working. Here's my parser setup for server side:
(pathom-connect/connect-plugin
 {::pathom-connect/register
  [cledgers-fulcro.resolvers/resolvers
   cledgers-fulcro.mutations-server/mutations]})
In the browser, in Developer Tools/Network, on the request, in preview I'm getting:
["^ ", "~$cledgers-fulcro.mutations-client/add-transaction",…]
0: "^ "
1: "~$cledgers-fulcro.mutations-client/add-transaction"
2: ["^ ", "~:com.wsscode.pathom.core/reader-error",…]
0: "^ "
1: "~:com.wsscode.pathom.core/reader-error"
2: "class clojure.lang.ExceptionInfo: Mutation not found - {:mutation cledgers-fulcro.mutations-client/add-transaction}"
As you can see, the mutations namespace on client-side (cljs) is cledgers-fulcro.mutations-client and on server-side (clj) it's cledgers-fulcro.mutations-server. Is it ok to have them named differently?
#2020-08-1521:08souenzzo@ballpark cledgers-fulcro.mutations-client/add-transaction is a pc/defmutation ? I usually use different namespaces for client and server mutations. My client mutations I keep next to defsc's#2020-08-1521:14Frank Henard@souenzzo, Thanks for looking at this! No cledgers-fulcro.mutations-client/add-transaction is a com.fulcrologic.fulcro.mutations/defmutation cledgers-fulcro.mutations-server/add-transaction is a com.wsscode.pathom.connect/defmutation#2020-08-1521:17souenzzoi usually do something like my-app.entity/operation << clj only, pc/defmutation here my-app.client/ui* << cljs ony, fm/defmutation referencing the full-symbol * my code organization of UI's usually a mess. I'm still working on it.#2020-08-1521:28lgessler@souenzzo how do you handle full-stack mutations? sounds like you're saying you transform the ast in the client mutations (remote ...) section to point at the pc mutation?#2020-08-1521:31souenzzoNo no. Mutations don't need to live in the symbol's namespace You can use app.entity/operation << mutation name app/server.clj << server
(pc/defmutation ...
  {::pc/sym 'app.entity/operation}
  ...)
app/client.cljs << client
(fm/defmutation app.entity/operation
  ...)
I use like that. Sometimes the server part matches the symbol name with the current namespace. But it's not a rule
#2020-08-1521:39lgessleroh interesting... but then in your client-side mutation, do you also do UI-related work in there or do you keep that in a separate mutation?#2020-08-1521:40souenzzoI do UI-related work#2020-08-1521:40lgessleri guess i'm anticipating that if you ever want to carry out app.entity/operation in two different UI components i'm not sure if doing that more than once would be ok?#2020-08-1521:40lgesslerdeclaring a fulcro mutation, i mean#2020-08-1521:42souenzzoI already "duplicate" a mutation, op-a op-b because in client are 2 distinct operations but in server, both do the same thing. It's not common#2020-08-1521:43souenzzoIt can also generate some cool analytics data: "90% of our users use add-comment-inline, and just 10% add-comment-menu"#2020-08-1521:44souenzzoAs we have pc/alias for resolvers, we can do for mutations (not sure if it exists ATM, but it CAN exists)#2020-08-1521:46lgesslerhm ok, so if we had another place in the client app/client/my-widget.cljs what would the mutation look like if we wanted it to also trigger a remote app.entity/operation?#2020-08-1521:46lgessleri think you're saying it'd be named something different but would still trigger the same remote somehow?#2020-08-1521:49souenzzoMy "mindset" is: the client send what it needs, server do what need to be done. When I'm developing the frontend, I don't care if the attribute/mutation exisist, i just "request" it. Then I go to backend and implement what the client needs It let frontend simpler and keep all complexity on backend#2020-08-1521:49Frank HenardHere's my mutations-server. I added ::pathom-connect/sym 'cledgers-fulcro.mutatinos-server/add-transaction, but it still doesn't appear to be registered.
(ns cledgers-fulcro.mutations-server
  (:require [clojure.pprint :as pp]
            [com.wsscode.pathom.connect :as pathom-connect]))

(pathom-connect/defmutation add-transaction [env params]
     {::pathom-connect/sym 'cledgers-fulcro.mutations-server/add-transaction}
     (let [_ (pp/pprint {:mutations-server {:env env
                                            :params params}})]
       {:cledgers-fulcro.entities.transaction/id 99}))

(def mutations [add-transaction])
#2020-08-1521:51souenzzoWhere you are looking to check if it's "registered" @ballpark?#2020-08-1521:52Frank HenardIt's not hitting the pprint, I'm still getting this response
["^ ", "~$cledgers-fulcro.mutations-client/add-transaction",…]
0:Ā "^ "
1:Ā "~$cledgers-fulcro.mutations-client/add-transaction"
2:Ā ["^ ", "~:com.wsscode.pathom.core/reader-error",…]
0:Ā "^ "
1:Ā "~:com.wsscode.pathom.core/reader-error"
2:Ā "class clojure.lang.ExceptionInfo: Mutation not found - {:mutation cledgers-fulcro.mutations-client/add-transaction}"
#2020-08-1521:53souenzzoYou need to use ::pc/sym 'cledgers-fulcro.mutations-client/add-transaction.#2020-08-1521:53Frank HenardHere's where I'm "registering" šŸ™‚ it:
(def pathom-parser
  (pathom/parser {::pathom/env {::pathom/reader [pathom/map-reader
                                                 pathom-connect/reader2
                                                 pathom-connect/ident-reader
                                                 pathom-connect/index-reader]
                                ::pathom-connect/mutation-join-globals [:tempids]}
                  ::pathom/mutate pathom-connect/mutate
                  ::pathom/plugins [(pathom-connect/connect-plugin
                                     {::pathom-connect/register
                                      [cledgers-fulcro.resolvers/resolvers
                                       **cledgers-fulcro.mutations-server/mutations**]})
                                    pathom/error-handler-plugin]}))
#2020-08-1521:53Frank Henarddouble earmuffs are just pointing it out#2020-08-1521:54souenzzoand be sure to reload the namespaces/the http server (if needed)#2020-08-1521:54souenzzoBut basead on what you send, the mutation names do not match.#2020-08-1521:56Frank HenardThat did it, thanks!#2020-08-1612:30yendaI'm trying to use pathom viz but I keep getting
WARN [taoensso.sente:1287] - Chsk is closed: will try reconnect attempt (14) in 1000 ms
Waiting for channel to be ready 1000
Waiting for channel to be ready 1000
#2020-08-1612:30yenda• I followed the readme and added the connector, even tried to use a parser like in the readme instead of a parallel parser#2020-08-1612:30yenda• electron app is opened, I tried the app image and I tried a version built from the repo#2020-08-1612:32yenda• I build with shadow on node target and had the following errors after adding the connector plugin: The required namespace "com.wsscode.async.processing" is not available, it was required by "com/wsscode/pathom/viz/ws_connector/impl/sente_cljs.cljs".Insufficient com.taoensso/encore version, you may have a dependency conflict: see https://goo.gl/qBbLvC for solutions.`#2020-08-1612:32yendawhich I fixed with these imports:
[com.wsscode/pathom-viz-connector "1.0.3"]
                [com.taoensso/encore "2.122.0"]
                [com.wsscode/async "1.0.11"]
#2020-08-1612:38yendaThere seem to be an error in the readme with the cond-> it's suppose to be cond->>#2020-08-1612:38yendahttps://github.com/wilkerlucio/pathom-viz-connector#2020-08-1614:36yendaI pathom viz logs all I have is
INFO [com.wsscode.node-ws-server:134] - Websocket Server Listening on port 8240
INFO [com.wsscode.node-ws-server:73] - Starting express
#2020-08-1708:42yendapathom-viz ws seems to be working:
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "b5ccd0dbd1d4f5d1d0c6dec1dac5"}, :content ("[emailĀ protected]")}
there is an expected error in the server since I just connect with no params:
WARN [com.wsscode.node-ws-server:60] - Unhandled request: %s /.websocket
ERROR [taoensso.sente:628] - Client's Ring request doesn't have a client id. Does your server have the necessary keyword Ring middleware (`wrap-params` & `wrap-keyword-params`)?: {:websocket? true, :websocket #object[WebSocket [object Object]], :response nil, :body #object[IncomingMessage [object Object]], :query-params {}, :form-params {}, :params {}}
#2020-08-1708:43yendaI tried the simplest possible connection on cljs side with:
(let [{:keys [chsk ch-recv send-fn state] :as res}
      (sente/make-channel-socket-client!
       "/chsk" ; Note the same path as before
       "not needed"
       {:host "localhost" :port 8240 :type :auto ; e/o #{:auto :ajax :ws}
       })]
(println :res res)
  (def chsk       chsk)
  (def ch-chsk    ch-recv) ; ChannelSocket's receive channel
  (def chsk-send! send-fn) ; ChannelSocket's send API fn
  (def chsk-state state)   ; Watchable, read-only atom
  )
#2020-08-1708:43yendano reaction on server side and the ws stays in closed state#2020-08-1709:07yendaI noticed in res that the url has no host#2020-08-1709:07yenda
(let [;; Not available with React Native, etc.:
                 win-loc  (enc/get-win-loc)
                 path     (or path (:pathname win-loc))]

             (if-let [f (:chsk-url-fn opts)] ; Deprecated
               [(f path win-loc :ws)
                (f path win-loc :ajax)]

               (let [protocol (or protocol (:protocol win-loc) :http)
                     host     (if port
                                (str (:hostname win-loc) ":" port)
                                (do  (:host     win-loc)))]
                 [(get-chsk-url protocol host path :ws)
                  (get-chsk-url protocol host path :ajax)])))
#2020-08-1709:08yendaunless I'm misunderstanding it looks like sente ignores the host and uses win-loc instead?#2020-08-1709:27yendaLooks like there is a bug in sente that is fixed in the next version, so I tried the newer version but it still doesn't connect#2020-08-1709:27yendaI am running this in nodejs#2020-08-1709:35yendaok looks like I had to yarn add websocket to my project#2020-08-1723:56wilkerluciodone, also fixed the example to use cond->>, thanks for pointing those out#2020-08-1720:25souenzzo@ballpark // @lgessler fulcro now has a function fm/with-server-side-mutation that can be used to "rename" the mutation before send it to remote#2020-08-1720:50lgesslervery useful, thanks!#2020-08-1807:17nivekuilthis is probably a micro optimization but all my batch resolvers end like this:
(if (> (count input) 1)       (pc/batch-restore-sort {::pc/inputs ids ::pc/key :view/id} result)       result)
is there a better way of doing this pattern?
#2020-08-1807:52ak-coramI have a macro that takes a sort key and generates this (and is also doing some async error handling stuff)#2020-08-1814:58mischovHas anybody used any kind of permissions based access control with pathom? If so, was it based on a library or framework, or did you roll your own? Anybody have any examples?#2020-08-1817:41lgesslerthere's no library for this afaik, some discussion of this here: https://github.com/souenzzo/eql-style-guide/issues/4#2020-08-1818:23lilactownwhere’s the best place to look for implementing unions?#2020-08-1820:51wilkerluciohttps://wilkerlucio.github.io/pathom/v2/pathom/2.2.0/connect/resolvers.html#_union_queries#2020-08-1821:04lilactownty!#2020-08-1821:05lilactownit doesn’t look the the source code in the docs for that demo, matches the demo query#2020-08-1821:10wilkerlucioups#2020-08-1821:10wilkerluciofixing it now#2020-08-1821:10lilactownšŸ™:skin-tone-2: thanks!#2020-08-1821:13wilkerluciofixed šŸ‘#2020-08-1818:35lilactownalso, what’s the best way to manually return a ā€œnot foundā€ value? return ::p/not-found? or is there a function I should call?#2020-08-1820:07souenzzo@lilactown for not-found: do not return the key. Resolvers "maybe" return the keys in output. If none, it will be marked as not-found#2020-08-1820:51wilkerlucio@lilactown you can return ::p/not-found directly as a way to short-circuit the thing, the difference will have in cases where there are multiple paths for that, that said, the preferred way is like @souenzzo said, dont return the key#2020-08-1820:56lilactownI’m not sure how to use the terminology correctly yet, so apologies if this is verbose. I have a resolver that I want to return a not-found value from#2020-08-1821:00lilactownto reuse my folders example (because it is the same thing I’m working on šŸ˜„ ) :
(pc/defresolver folder-tree-resolver
  [{:keys [db]} {:keys [::folder/id]}]
  {::pc/input #{::id}
   ::pc/output [{::tree (vec (concat file-keys folder-keys))}]}
  (let [files (list-files db)
        folders (list-folders db)
        children (filter (comp #(= % id) ::folder/parent) (concat files folders))]
    {::tree (if (empty? children)
              ::p/not-found ;; no children for this particular node
              (vec children))}))
#2020-08-1821:02lilactownso in this case, when a ::folder/id key exists in the context it will look up to see if any files or folders declare it a parent#2020-08-1821:02lilactownin the case where it found no files or folders, I think it’s better to return ::p/not-found (and have it elided by the plugin) then an empty vec#2020-08-1821:02wilkerluciomaybe something like this if I got right:
(pc/defresolver folder-tree-resolver
  [{:keys [db]} {:keys [::folder/id]}]
  {::pc/input #{::id}
   ::pc/output [{::tree (vec (concat file-keys folder-keys))}]}
  (let [files (list-files db)
        folders (list-folders db)
        children (filter (comp #(= % id) ::folder/parent) (concat files folders))]
    (if (seq children)
      {::tree (vec children)})))
#2020-08-1821:03wilkerluciosorry#2020-08-1821:03wilkerluciofixing itt#2020-08-1821:03wilkerluciook, should be right now#2020-08-1821:03lilactownahh so returning no map would be akin#2020-08-1821:03wilkerlucioyeah, or empty map#2020-08-1821:04lilactowncool! I figured it was strange to return the keyword. that helps, thanks!#2020-08-1821:05wilkerluciosmaller tip, you can use filterv on the children#2020-08-2018:00felipethomeHi! Is it ok to go from 2.2.30 to 2.3.0-alpha9? I guess the alpha is related to reader3 and nothing related to old functionalities right? I’m testing here and everything seems ok, but I want to confirm that#2020-08-2018:37wilkerlucioyes, its ok to change, there were few other internal changes, if you find any issue please let me know#2020-08-2018:54felipethomethanks, :thumbsup:#2020-08-2019:08souenzzo@U066U8JQJ reader3 should be used?#2020-08-2101:29wilkerlucioat your own risk, good chance of some edge case bugs, needs maturing#2020-08-2101:29wilkerluciobut good for experimentation#2020-08-2018:17lilactownis there an easy way to embed the query explorer in my front-end app? or deploy it w/ my pathom service?#2020-08-2018:38wilkerlucioyes: https://roamresearch.com/#/app/wsscode/page/RG9C93Sip#2020-08-2213:30yendais such query invalid?
[{[:video/id "faf7789f-d3e2-11ea-9add-02a4a06c46e9"]
  [:video/placeholder-url]}
 {[:video/id "faf7789f-d3e2-11ea-9add-02a4a06c46e9"] [:creator/id]}]
#2020-08-2213:31yendashouldn't the parser merge the outputs instead of overwriting them (which I assume it does since it responds with one or the other)?#2020-08-2215:09souenzzo@yenda it's not a invalid query But both results end up in the same ident. It will not merge
[{([:video/id "faf7789f-d3e2-11ea-9add-02a4a06c46e9"] {:pathom/as :a})
  [:video/placeholder-url]}
 {([:video/id "faf7789f-d3e2-11ea-9add-02a4a06c46e9"] {:pathom/as :b})
  [:creator/id]}]
#2020-08-2215:10souenzzoOr using placeholders
[{:>/a [{[:video/id "faf7789f-d3e2-11ea-9add-02a4a06c46e9"]
         [:video/placeholder-url]}]}
 {:>/b [{[:video/id "faf7789f-d3e2-11ea-9add-02a4a06c46e9"]
         [:creator/id]}]}]
#2020-08-2218:05yendaI have the following attributes:
[:comment/created-on
                :video/id
                :user/id
                :comment/id
                :comment/text
                :comment/deleted?
                :comment/mentioned-id]
#2020-08-2218:06yendaboth user/id and comment/mentioned-id are user-ids but I don't see how I can get more info about the mentioned-id with one query#2020-08-2218:06yenda
[{[:comment/id "b214304f"] 
                                                             [{:>/mentioned-user [:comment/mentioned-id :user/username]} :comment/created-on 
                                                              {:>/user [:user/id :user/username]}]}]
#2020-08-2218:06yendathis doesn't work and picks the username of user/id in both placeholders#2020-08-2218:10yendaI have an alias for mentioned-id (pc/alias-resolver2 :user/id :comment/mentioned-id))#2020-08-2220:02wilkerlucio@yenda that's a modeling issue, because you have multiple paths with different input values, so which path it chooses is unpredictable (not a problem when paths are consistent, but its not this case), a way out is to instead of making those alias, make that a relationship, in this way:#2020-08-2220:03wilkerlucio
(pc/defresolver comment-mentioned [_ {:keys [comment/mentioned-id]}]
  {::pc/input #{:comment/mentioned-id}
   ::pc/output [{:comment/mentioned [:user/id]}]}
  {:comment/mentioned {:user/id mentioned-id}})
#2020-08-2220:04wilkerluciothen, you can query:
[{[:comment/id "b214304f"]
  [{:comment/mentioned [:user/username]}
   :comment/created-on
   {:>/user [:user/id :user/username]}]}]
#2020-08-2220:05wilkerluciothe idea is created a separated context for the specific (mentioned) entry, this way it doesn't conflict with the user in the parent#2020-08-2220:06yendanice thanks#2020-08-2321:27wilkerlucioHello everyone, just released [com.wsscode/pathom "2.3.0-alpha10"], this fixes the core.async warnings from to-chan and onto-chan, thanks @eoliphant for bringing it up#2020-09-0219:41Josh WoodHaving trouble getting a Pathom parser to connect to mssql server via next.jdbc.. Has anyone here accomplished this?#2020-09-0222:07wilkerlucio@joshuawood2894 can you tell more about the specific issue you having? Pathom has no direct integration with SQL, but you can write resolvers and make the calls to the database#2020-09-0300:11Chris O’Donnell@joshuawood2894 here's an example of how to use pathom with postgres if it's helpful: https://chrisodonnell.dev/posts/giftlist/backend_persistence/ Not sure how different mssql might be.#2020-09-0314:21Josh Wood@codonnell Thanks for the example! I'll check that out.#2020-09-0314:32Josh Wood@wilkerlucio So I'm trying to create a Fulcro/Pathom application with a mssql db starting with the Fulcro template. In the template's Pathom parser, it has a /env-wrap-plugin function that I'm trying to use to connect the db via next.jdbc. This is how I'm handling it at the moment.
(def db {:dbtype "mssql"
         :dbname ""
         :host ""
         :port ""
         :user ""
         :password ""})

(def con (jdbc/get-connection db))
(def ds (jdbc/get-datasource db))
.
.
.
::p/plugins [(pc/connect-plugin {::pc/register my-resolvers})
             (p/env-wrap-plugin (fn [env]
                                  ;; Here is where you can dynamically add things to the resolver/mutation
                                  ;; environment, like the server config, database connections, etc.
                                  (assoc env
                                    :db ds
                                    :connection con)))
This code compiles and I've tested to see if con and ds succeed in the repl.. no errors. The confusing part to me is when I write a resolver that matches content in my db and try to use Fulcro Inspect to run a query, the parser doesn't seem to wrap the connection e.c. the queries don't come back at all, not even with a 'not found'. Any suggestions? I am new to this architecture so there's surely something fundamental that I'm mission here.
#2020-09-0314:37Josh Wood*missing#2020-09-0322:51psdpHello, need some help here. I set ::pc/key-process-timeout to 300000 in parallel-parser settings, but it seems to have no effect. Still getting error Parallel read timeout timeout: 60000#2020-09-0402:10souenzzo@psdp did you put the ::pc/key-process-timeout at (p/parallel-parser {::pc/key-process-timeout ...}) or at env (parser (assoc env ::pc/key-process-timeout ...) ....) ?#2020-09-0402:46psdp@souenzzo Thanks for the hint! Problem solved now I put it at env. And I set a wrong namespaced keyword, it should be ::pp instead of ::pc#2020-09-0421:33Frank HenardI'm trying to do a next.jdbc.sql/insert! in a pathom-connect/defmutation. It appears that it's not running the insert, and I'm not seeing any exceptions. Does this have something to do with the fact that I'm using http-kit which is asynchronous, and next.jdbc is synchronous?#2020-09-0423:16souenzzo@ballpark try to call the opertion on your REPL something like (parser env '[(app/insert-in-db {:a 42})])#2020-09-0600:53sihingkkhi, had anyone experience with using pathom together with neo4j (or any other graph db? )#2020-09-0817:58lilactowndoes pathom support automatic recursion, a la ... in datomic pull syntax?#2020-09-0818:25wilkerlucioyes#2020-09-1119:35jimberlageIs there a particular reason why defresolver/defmutation aren't wrapped with #?(:clj) at the moment? It prevents pathom from compiling under shadow-cljs, but I was wondering if there's a reason for doing it this way that I'm missing#2020-09-1119:46souenzzo@jimberlage you can use pathom both on server and client#2020-09-1119:48jimberlageI know - it shouldn't prevent functionality on the server. The other macros are wrapped in #?(:clj), which plays nice with clojurescript. It's only defresolver/defmutation that are unwrapped, which causes compiler errors#2020-09-1119:50souenzzoWhere? https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/connect.cljc#L1541#2020-09-1119:50jimberlageHere's an example of a macro in cljc that works well when compiling under shadow-cljs: https://github.com/wilkerlucio/pathom/blob/2c707bbcad45d37250a89332ae889865ab8498ca/src/com/wsscode/pathom/trace.cljc#L53#2020-09-1119:50jimberlagehttps://github.com/wilkerlucio/pathom/blob/2c707bbcad45d37250a89332ae889865ab8498ca/src/com/wsscode/pathom/trace.cljc#L9#2020-09-1119:51jimberlageIf the code were updated to do the same for the other macros, pathom would work under shadow-cljs no problem#2020-09-1119:55wilkerlucio@jimberlage hello, can you open a issue for it? or if you like to change and send a PR, welcome as well šŸ™ #2020-09-1119:56jimberlageI already have a PR open for it, if you want to check it out - https://github.com/wilkerlucio/pathom/pull/172#2020-09-1119:57wilkerlucio:man-facepalming: my bad, checking it now#2020-09-1213:53wilkerluciomerged āœ…#2020-09-1119:57jimberlageReally appreciate your work on pathom, btw!#2020-09-1316:34kennyIs there anything to be aware of if I were to call the parser within a resolver?#2020-09-1317:50wilkerlucioMost things should work, but understanding how pathom works can surely helps, a few points that quickly come to my mind: - cache is shared, so if you env up hitting cached resolvers, pathom will use the cache - adds a cost of more planning, which just means you are doing some of the parser work again (convert query to ast, create a plan for each attribute you ask, plugin infra to run, etc...)#2020-09-1510:24Thomas MoermanI was wondering about this too, is calling a parser from within a mutation the correct approach for e.g. resolving some permission keywords that need to be taken into account? Or is there a better approach to resolve some contextual information during a mutation?#2020-09-1514:49kenny@U052A8RUT Perhaps your use case is more specific but in our mutations we'll typically return a map that has, at least, a /id attribute that a pathom resolver takes as an input. Pathom will then resolve any additional keys through the regular facilities.#2020-09-1518:31Thomas MoermanMy case is different: the idea is to call the parser to use the existing resolvers to resolve additional (e.g. permission) info that might prevent the mutation from executing. So it's not a mutation join using an /id return value, as you mention.#2020-09-1518:49kennyCould you alter the query prior to calling the parser?#2020-09-1807:23fjolne@U052A8RUT it’s generally fine, but you should beware of calling parser with the current env, as it can be modified via open idents by the user [1]. there’s some discussion on auth best practices [2] [1] https://github.com/souenzzo/eql-style-guide#security [2] https://github.com/souenzzo/eql-style-guide/issues/4#2020-09-2208:15Thomas MoermanThanks, i'll look into it.#2020-09-2209:35Thomas MoermanOk I think I understand it now, that's indeed a subtle vulnerability that I did overlook tbh... many thanks for pointing that out#2020-09-1420:32Ī»ustin f(n)What options are there in speeding up the compute-plan step? Is it possible to provide a way to cache the compute plan for common queries?#2020-09-1513:49henrikIs it possible to use back navigation in Pathom? (I’m using https://github.com/wilkerlucio/pathom-datomic). I.e.,
[{[:realm.schema/name :u.fluent/task]
  [{:realm.node/_schema
    [:realm/id
     {:realm.node/root [:realm/id]}]}]}]
#2020-09-1514:03avocadeOr do we need to create manual resolvers for this kind of thing? Would be pretty awesome if the pathom-datomic plugin could just pass that through…#2020-09-1513:50henrikDo I need to install realm.node/_schema manually for it to be recognized perhaps?#2020-09-1516:15wilkerluciono, back references are currently unsupported, if you like to add this I'll take a PR, what needs to change is to also generate the back link properties in the index: https://github.com/wilkerlucio/pathom-datomic/blob/f8a7d25e39cb3be13362eb8016402610f85f8517/test/com/wsscode/pathom/connect/datomic_test.clj#L770-L816#2020-09-1518:44avocade@U066U8JQJ thanks for the info! let’s get to work then I guess @U06B8J0AJ 🌟 šŸš€ #2020-09-1607:16henrikFigured as much @U066U8JQJ, creating a manual parser works quite well, so it’d be great to have them generated. We’ll have a look at it and see if we can’t contribute something.#2020-09-1721:09wilkerlucio#2020-09-1801:37nivekuilcan dynamic resolvers reduce the work done in a resolver? e.g. if you have a resolver with ::pc/output [:first-name :last-name] and you only query for [:first-name], can pathom modify the resolver logic so it doesn't query the db for :last-name too?#2020-09-1801:45wilkerlucio@kevin842 you mean a way to check in the resolver what query was done to him, to maybe avoid providing parts of it?#2020-09-1801:45nivekuilyeah#2020-09-1801:45wilkerlucioyes, that's something dynamic resolvers can do, and what you said is how I imagine the implementation of things like SQL drivers could be#2020-09-1801:46wilkerluciothe new planner gives to the dynamic resolver the exact query it should fulfil#2020-09-1801:46wilkerlucio(based on the user request)#2020-09-1801:56nivekuilsounds perfect :) I'm using pathom with crux and I guess that means there's no cost to defining a single resolver for a big document, so pathom can describe a de facto schema for crux with a 1:1 mapping between resolvers and documents#2020-09-1802:25cjmurphyPathom 3 sounds great. You did say 'great time to bring up new ideas'. So the idea, which I know I've mentioned before, is for mutations to have inputs, so they work more like resolvers. The usual work around for this is to call the parser that's in the env again. Having inputs as 'first class' for mutations would make mutations look better, and surely improve other things too, like performance??#2020-09-1804:41wilkerluciohello, I think this should be done in plugin lang, or using transforms, that's because adding the processing of parsing params as inputs can never be faster than just providing raw params, so I consider that mutations need to always have the most lightweight version possible available#2020-09-1804:46wilkerluciobut sure, I think providing those as built-ins could be a easy way to make it accessible, or maybe I'm missing something, what I said makes sense to you?#2020-09-1805:36cjmurphyYes pretty much. A second arg to the mutation. Usually the mutation would have only one arg as now. So the performance stuff can perhaps be done automatically from what the user indicates (one or two args)...#2020-09-1807:48henrikSmart maps are intriguing. Seems a bit like the concept of a lazy seq, applied to a map.#2020-09-1815:25eoliphant+1 on the smart maps. I always thought Plumatic’s Graph was a super cool, but it doesn’t work with namespaced keywords, etc. seems a super cool fit with pathom#2020-09-1815:33wilkerlucioyup, I feel like you guys, and I also believe this is a much easier way to use pathom for newcomers, so they don't have to learn anything about EQL but can still leverage the resolvers engine#2020-09-1815:58henrikThat’s a solid point#2020-09-1815:39Chris O’DonnellSmart maps seem pretty neat, but I think I would prefer to use it via a different API from the normal clojure.core get. When I'm reading code and I see (:my.ns/foo m), I have an expectation that the operation will happen immediately. If (:my.ns/foo m) were to cause the parser to go out and make an API call to a microservice to get data (for example), I think that starts to tread on the principle of least surprise. Something like (pathom.smart-map/get m :my.ns/foo) is a bit more verbose, but also more explicit.#2020-09-1815:41Chris O’DonnellJust 2 cents from someone with far less pathom experience. :)#2020-09-1815:43wilkerlucio@codonnell makes total sense, I think this decision between transparent or not can very depending on the use case. for example, I imagine smart maps can be used to make adapter layers, so you can swap names, in this case a user may prefer to have it transparent as a map, and don't care if its immediate or lazy. but if you are dealing with heavy things like API calls, and the surprise effect is problematic, then would be nice to make it explicit. and I think Smart Maps should support both cases, and this can be a matter of how to configure the smart map (to make lazy calls implicit or explicit)#2020-09-1815:46wilkerlucioanother interesting configuration that will happen is how the smart maps should respond to (keys smart-map), so far I imagine two options here: 1. cached - this is the current way, keys will respond with the keys that are cached on the map, so the lookup is constant and predictable 2. reachable - this will respond with all possible keys (considering the index and current context), interesting in some cases but needs to be used with care, because a (into {} smart-map) could trigger a lot of resolvers#2020-09-1815:46Chris O’DonnellOh yeah, that is an interesting wrinkle.#2020-09-1815:49Chris O’Donnell2. could have interesting interactions with quite a bit of the clojure sequence API#2020-09-1815:52Chris O’DonnellIf (keys smart-map) returned something different from the tuples in (seq smart-map), for example, (into {} smart-map) could return something different from (reduce (fn [m k] (assoc m k (get smart-map k))) {} (keys smart-map)), which would be very surprising.#2020-09-1815:54Chris O’DonnellNow that I'm thinking more about it, I can see how it would be valuable to treat smart maps like maps so you could use clojure.core collection functions on them.#2020-09-1815:57Chris O’DonnellDon't want to come off to negative; I am pretty excited to see pathom 3 as it develops. šŸ™‚#2020-09-1816:05henrikKeeping in mind the goal of being approachable by new-comers, I think the path of least surprise would be the most sensible default, which to my mind would be get and keys behaving as you would expect them to behave when applied to a map. I.e., return the value, return all keys (does the map necessarily have to realise the values to return the complete set of keys?).#2020-09-1816:11wilkerlucio@henrik I agree on the path of least surprise, to me so far that is having keys listing only cached, and with transparent reads#2020-09-1816:11wilkerluciothe keys thing doesn't need to realize the value, but gets dangerous when doing any scan operation (like (into {} smart-map))#2020-09-1816:12henrikI’d have to disagree with that, since it kind of becomes mutable (and therefore surprising). Reading the map changes the output of keys.#2020-09-1816:13wilkerluciowe could have different results from keys and seq, but I'm not sure that is least surprising, I guess we need to experiment more to figure those things out#2020-09-1816:19henrikWell, scan operations being potentially expensive isn’t unheard of in Clojure. Any lazy construct will have pitfalls like that.#2020-09-1816:19wilkerlucioyeah, but I fear that it could hurt programs in quite bad ways, therefore I rather have the default to be safer in this case#2020-09-1816:24henrikI think the ā€œmutabilityā€ is the more dangerous problem in this case. We all have REPLs, and we can evaluate something and gauge the performance of it. But when you start talking about something that ā€œchanges in placeā€, like the output of keys following a get, you can easily get the wrong impressions of what’s happening just because you evaluated your forms in a particular order while developing (as in oops, you evaluated a get, the keys output is not going to look the same in production as it did in your REPL).#2020-09-1816:30wilkerlucioI can understand that, but I still more afraid of long runs, IME the graphs tend to get really big, and when they do, a single attribute may reach hundreds of resolvers, so even on the REPL a simple scan on a single attr could stop and break everything, this makes me feel this is a too dangerous operation to be the default, but as I said, from impressions so far, I'm down to change opinion depending on how the use cases unfold and we have a better understanding of the most common types of usage#2020-09-1816:32henrikIn that case, maybe it would be better to stay away from core entirely, since it would kind of redefine what those functions do anyway, and expose a custom set of functions to interact with them.#2020-09-1816:33henrikWith a disclaimer that I’m not at all sure, it’s certainly a tricky bunch of tradeoffs to juggle.#2020-09-1816:33wilkerluciowill really depend on each use case, I really like the transparent idea for conversion layers, and users that think better defaults should be different, you can write your own smart map constructs to change the defaults in your application context, do you think that's a reasonable solution for difference in opinions around the defaults?#2020-09-1816:34wilkerluciooh yeah, I agree its a tricky jungle, and I hope we can figure it out together what best practices around it may be (or even if its just a really bad ideia, haha)#2020-09-1816:36henrikIf ā€œbuild your ownā€ means just seeding some kind of reusable construction function with a config that flips a few switches, maybe. Otherwise, people are probably going to use the defaults or avoid it entirely and go ā€œclassicā€.#2020-09-1816:37wilkerlucioyeah, I mean writing something like this:
(defn my-smart-map [env context]
  (psm/smart-map 
    (merge {::psm/keys-mode ::psm/keys-mode-reachable} 
           env)
    context))
#2020-09-1816:42henrikIf you expect it to be common enough, you could shave away the env, context with a helper maybe:
(def my-smart-map (psm/make-smart-map {::psm/keys-mode ::psm/keys-mode-reachable}))
#2020-09-1816:45wilkerluciowe can use a partial in that:
(def my-smart-map (partial psm/smart-map {::psm/keys-mode ::psm/keys-mode-reachable}))
what makes hard to shave the env out is because you also need to provide indexes, if your app wants to use a single index for all smart maps, that's fine, but otherwise you will still want to have the env available to change (to use different indexes)
#2020-09-1816:46wilkerlucioand maybe different indexes will be the key for different usages, a simple index with mostly aliases and quick operations should be safe to allow all keys, but one with many heavy resolvers will want a different setting#2020-09-1816:57henrik
(defn make-smart-map [config]
  (fn [env context]
    (psm/smart-map (merge config env) context)))
?
#2020-09-1817:04wilkerlucioyeah šŸ‘#2020-09-1816:24pithylessWhat if the smart-map followed the conventions of Datomic Peer entities? keys would only return cached keys and you would have to do something akin to d/touch to realize everything.#2020-09-1816:25wilkerlucio@pithyless this is the direction I'm inclined to, sm also support (psm/load! smart-map eql-to-touch), which is similar the touch you are talking (but more specific, not getting everything, altough that could be an option)#2020-09-1816:27pithylessYeah, that sounds great. I'm all in favor of being explicit via EQL ;]#2020-09-1816:29pithylessOne could always support (psm/load! smart-map '[*]) if that was a common feature request.#2020-09-2120:58Michael J DorianHey, I'm giving Pathom a try and I noticed that I can't require com.wsscode.pathom.core and com.wsscode.pathom.connect because of No such var: async {:form async/onto-chan!, :file "com/wsscode/pathom/core.cljc", :column 13, :line 560} on version 2.3.0-alpha10. Changing it to 2.3.0-alpha9 seems to have solved the issue. Did I find a problem or am I maybe missing something?#2020-09-2121:23wilkerlucio@doby162 dependency issue, getting the wrong version of core.async, bump pathom to alpha-10, or bump core.async to latests#2020-09-2121:23wilkerluciooh, you got the problem on alpha-10? let me check here#2020-09-2121:24wilkerlucio(bumping core.async should fix it any case)#2020-09-2121:24wilkerlucioI checked, alpha-10 points to org.clojure/core.async 1.3.610, maybe you have some dep overriding this (or have core.async explicitly, with some older version)#2020-09-2121:26Michael J Dorianexplicitly requiring core.async's latest version fixed it, I guess one of the Luminus libraries must be asking for an older version? Thanks!#2020-09-2308:23psdpIs it possible to call the parser with arbitrary EQL query inside a resolver?#2020-09-2421:11Michael WHaving a weird issue:
(defattr id :network/id :int
  {ao/identity?  true
   ao/pc-input   #{:network/id}
   ao/pc-output  [:network/id :network/name :network/start :network/end :network/parent :network/site]
   ao/pc-resolve (fn [_ {:keys [network/id] :as input}]
                   #?(:clj
                      (transform-network-data (solid/network-id (str id)))))})

(defattr network-name :network/name :string
  {fo/field-label "Network Name"
   ao/required?   true
   ao/pc-input    #{:network/name}
   ao/pc-output   [:network/id :network/name :network/start :network/end :network/parent :network/site]
   ao/pc-resolve  (fn [_ {:keys [network/name]}]
                    #?(:clj
                       (transform-network-data (solid/network-name name))))})

(defattr all-networks :network/all-networks :ref
  {ao/target     :network/id
   ao/pc-output  [{:network/all-networks [:network/id]}]
   ao/pc-resolve (fn [{:keys [query-params] :as env} _]
                   #?(:clj
                      {:network/all-networks (mapv #(Integer/parseInt (:subnet_id  %)) (solid/network-all))}))})
With this in fulcro, I can run the query [:network/all-networks] and get back the full list of ids ok. I can run the query [{[:network/id 33] [:network/name]}] with no issues. But if I try to run a query like this: [{:network/all-networks [:network/name]}] I get an error about contains? not supported on type Integer:
com.wsscode.pathom.core/map-reader     core.cljc:  629
clojure.core/contains?                 core.clj: 1492
                               ...                               
java.lang.IllegalArgumentException: contains? not supported on type: java.lang.Integer
What am I doing wrong?
#2020-09-2421:21wilkerlucio@michael819 the issue is in the value of :network/all-networks, by looking at your code I guess its like this: {:netword/all-networks [1 2 3 4 ...]}, but that's wrong, it needs to be {:netword/all-networks [{:network/id 1} {:network/id 2} {:network/id 3} ...]}#2020-09-2421:22Michael WAhhh#2020-09-2421:29Michael W@wilkerlucio Perfect that fixed the issue.#2020-09-2815:04imreHey all. I was just going through some parts of the pathom code and found that there are lots of single-branch if s, if-let s etc. Is there some advantage of these in pathom's (or a more general) context over their when... counterparts?#2020-09-2815:43wilkerluciono, they are the same, its a preference thing, to me its just that if has half of the letters of when, but for a single branch they are equivalent#2020-09-2816:00imreMakes some sense I guess, thank you for the explanation šŸ™‚#2020-09-2909:24cjmurphyIt makes the code slightly harder to read, because it is unconventional. With an if I'm always looking for two branches. When I first saw it it really broke my concentration! At least now I know it is intentional. In my own code it would be a 'mistake', that sometimes does happen.#2020-09-2909:36imreYeah, we'd also request these to be changed in our own codebase.#2020-09-2913:17tvaughanI have this resolver:
(pathom-connect/defresolver session-id-resolver
  [{:keys [::pathom/parent-query]} {:keys [sessions/session-id]}]
  {::pathom-connect/input #{:sessions/session-id}
   ::pathom-connect/output [:sessions/session-id :sessions/user-id]}
  {:sessions/session-id :foobar
   :sessions/user-id 42})
When I make this query [{[:sessions/session-id :TODO] [:sessions/session-id :sessions/user-id]}] I get back {[:sessions/session-id :TODO] {:sessions/session-id :TODO, :sessions/user-id 42}}. Why do I get back :TODO instead of :foobar ? I'm using v2.2.31. Thanks
#2020-09-2913:20souenzzo@tvaughan once you do {[:a 42] [:a :b]} you already providing the value of :a, so it will call the resolver just to get the value of :b#2020-09-2913:24wilkerlucio@tvaughan pathom will never override a value that is already in the entity, when you start with the ident, it set the value of that attribute, even if new resolvers get new values, the original value will be kept. if you consider that in Pathom the identity is the data itself, changing the data is like changing the identity, which leads to inconsistent data (if many resolvers use the same input property, with different values)#2020-09-2913:24wilkerluciomakes sense?#2020-09-2913:26tvaughanSure, that makes sense. I'm not advocating for a different behavior. Thanks for letting me know this is expected @souenzzo and @wilkerlucio#2020-09-2913:42souenzzo@tvaughan see also: ::pc/mutation-join-globals It allow you to handle tempids For sessions, I use [{:app/current-session [:session/id :session/user-id ...]}]#2020-09-2913:42souenzzoI started this repo but it need some attention/commits https://github.com/souenzzo/eql-style-guide#2020-10-0423:38nivekuilwhat would it take to be able to return manifold deferreds from resolvers instead of core.async channels?#2020-10-0518:46wilkerlucioYou can do something similar to converting promises to channels wrapping one in another. In the end you need to provide a channel (a promise-channel preferably) to Pathom, some ways you can make it easier on you are: - create a ::pc/transform operation, so you can easely plug specific resolvers to accept manifold - if you wanna use it a lot, you can wrap a plugin using ::pc/wrap-resolve so you can detect manifold outputs and convert then to core.async channels#2020-10-0516:36njjWhat is the typical way of setting up a resolver where you might not know all the outputs from the api response something like:#2020-10-0516:37njj
(pc/defresolver some-resolver
                [__ {:keys [resource/id]}]
                {::pc/input #{:resource/id}
                 ::pc/output [:id]}
                (let [resource (get-resource-by-id id)]
                  resource))
#2020-10-0516:37njjwhere I could still query other parts of the response like: `[{[:resource/id 1] [:id :name]}]`#2020-10-0518:47wilkerlucio@njustice Pathom needs to know the output ahead of time, otherwise it can't figure the proper resolve to call an attribute#2020-10-0518:47wilkerlucioso you can't have that open#2020-10-0518:47wilkerluciodynamic resolvers can be a bit more cherry picking on it, but they still need to know all possibilities ahead of time#2020-10-0518:50Michael WYou sort of can have it open, if you build a resolver by id, but have it return all data, you can then use something like filter to pull out what you need. fulcro-rad-demo does it with attributes that have a name starting with "all-". But pathom won't be able to query by any of that output unless it's in the output list.#2020-10-0518:52njj@wilkerlucio in my example, I get back the :name w/o including it in the pc/output#2020-10-0518:56njjI.e. `{[:resource/id 1] {:name "foo", :id 1}}`#2020-10-0518:56wilkerlucio@njustice the problem is that this makes it a an implicit dependency, in a way its like getting it by accident. the problem is that if you don't ask for the id, you will never get :name. also, I like to discourage you from having unqualified attributes in the system (input or output) due that it will easaly collide with other things#2020-10-0518:57njjso for any resource I have, if I want to use a field from it, I need to include it in the output#2020-10-0614:57tvaughanOr stuff them all into a hash-map under one key. The contents of the hash-map under this key become arbitrary#2020-10-0518:57wilkerlucioyes, you can think of it as your schema in a way#2020-10-0518:57njjgotcha#2020-10-0623:26λustin f(n)Are there any EQL client libraries that are more standalone than Fulcro itself? A-la how there are GraphQL client libraries for a number of web frameworks including React? https://www.apollographql.com/docs/react/#2020-10-0623:29λustin f(n)I am trying to transition to a Graph query paradigm for my team, but we have a number of React front ends we support. I would like to be able to take advantage of EQL/Pathom for them as well without having to commit those projects to transitioning to Fulcro entirely.
#2020-10-0623:31Ī»ustin f(n)As far as I can tell, without a way to 'partially transition' using a client, I will have to use GraphQL instead. But in working through that approach I have found it just isn't as powerful as Pathom's model.#2020-10-0700:51souenzzo@austin021 I work on a internal library that allow you to describe pages from EQL queries, using SSR generated pages Also render mutations as forms etc...#2020-10-0700:52souenzzoif you are interested on something like it let me know. then I will try to port it out from my internal project#2020-10-0701:27mruzekw@austin021 You could setup a GraphQL backend and front it with a Pathom instance where you want to use ClojureScript with. It’s also possible to only use the DB/network layer of Fulcro as seen in this video with Fulcro + Re-frame: https://www.youtube.com/watch?v=ng-wxe0PBEg#2020-10-0706:26nivekuilhas anyone found a reason to not use elide-special-outputs-plugin?#2020-10-0712:23souenzzomany pathom behaviors are "historical"#2020-10-0712:24souenzzoand the approach is never break what is working. If someone did a application using ::p/not-found, it still works But every new pathom app should use this plugin#2020-10-0715:57markaddlemanI'm wondering if I'm doing something wrong. I'm integrating Pathom into an existing application which has its own notion of keys for data. It's pretty straightforward to encode the existing application keys into idents. For example, here is an ident for an "event":
[:application.event/id
          {:application/id {:appKey "vI7INwaYRrmRg20PrWM1UQ", :portfolio/id {:portfolioKey "Z8X06QFpRXiu7BCyBg2xOg"}},
           :event          "Back to Start"}]
The resolvers which retrieve data from our physical stores need different bits from the idents. For example, my resolver to retrieve "event attributes" looks like this:
(pc/defresolver application-event-attributes [env input]
  {::pc/input  #{:portfolioKey :appKey :event}
   ::pc/output [{:application.event/attributes [...]}]}
  {:application.event/attributes ...})
Note the physical resolver doesn't take :application.event/id directly. My idents are basically hierarchically nested structures that serve as keys into application data. In our domain, portfolios have applications, applications have events and event have attributes. So, an event ident nests the application ident and the application ident nests the portfolio ident. Following this pattern, an event attribute ident nests the event ident, blah blah... To make this work, I end up writing a bunch of resolvers that explain to Pathom how to decompose my idents. So, I have a bunch of resolvers that look like this:
(pc/defresolver application-event-id->application-id-and-event [env {:application.event/keys [id]}]
  {::pc/input  #{:application.event/id}
   ::pc/output [:event :application/id]}
  id)
Is it common to take a nesting approach to idents? Does Pathom have a more direct way of understanding the idents structures so I don't have to write a bunch of these decomposing resolvers? Am I fundamentally approaching this the wrong way?
#2020-10-0913:50wilkerluciohello @markaddleman, doing decomposition resolvers is fine if you wanna give a "name" to a composite structure, one example case that I find interesting to do it is for leiningen style deps, breaking something like :library/path [org.clojure/core.async "1.3.610"] to the attributes :library/symbol :library/version, so this is a legit usage. In your case I see that the composed structure already has all the keys, so you can instead using something like:#2020-10-0913:51wilkerlucio
[{([:portfolio/id 123] {:pathom/context {:application/app-key "..."}})}]
#2020-10-0913:52wilkerlucioanother suggestions from what I'm seeing in your code: 1. I see some unqualified keywords, I strongly recommend to avoid those, they may clash with other names too easely 2. try to think more flat, I see nestings like :portfolio/id {:portfolioKey "Z8X06QFpRXiu7BCyBg2xOg"}, I suggest to avoid those wrappings and just make long attributes that have context-free meaning#2020-10-0913:52wilkerlucioa common approach is at the edges (when you are hitting some actual service or database), make an adapter and convert any unqualified name to fully qualified ones#2020-10-0914:29markaddlemanThanks. The point of wrapping :portfolioKey with :portfolio/id was to avoid clashing and remain compatible with the backend. I hadn't thought of stripping the namespaces. Good point!#2020-10-1020:40ziltiWhat is the approach to test a pathom mutation? I have a mocked env, I basically just need a way to synchronously call the mutation "like a function"#2020-10-1022:35wilkerluciothe mutation is a map, extract ::pc/mutate from it, then call it#2020-10-1023:10ziltiOh, nice! šŸ™‚ Thank you!#2020-10-1116:41yendais there a way to specify the union-path with a defresolver? i.e I want to do the union query on the value of the :type key and not some specific keys in the map#2020-10-1120:39yendaSo if anyone is looking for a solution some day: my problem was that not only I have a union query based on the value of a specific key :notification/type, I also want to remove some of the results based on some of the values of some key (ie video/deleted?, user/blocked?) I also want pagination, which I do using the following output:
[:pagination/edges [{:follow [:notification/type
                                             :notification/timestamp
                                             :user/id]
                     :comment [:notification/type
                                              :notification/timestamp
                                              :comment/text
                                              :user/id
                                              :video/id
                                              :video/deleted?]...
   :pagination/has-prev?
   :pagination/has-next?
   :pagination/first-cursor
   :pagination/last-cursor]
so to achieve this I used 2 resolvers, one that outputs [{:notifications [:raw-notifications]}] and one that outputs the schema above the second resolver takes the raw-notifications as input and uses join-seq-parallel to resolve the raw notifications so that the resolver can filter on the values initially I wanted to use one resolver but for some reason I couldn't get join-seq-parallel to work, I would get
[{:pagination/first-cursor :com.wsscode.pathom.core/not-found, :pagination/has-next? :com.wsscode.pathom.core/not-found, :pagination/last-cursor :com.wsscode.pathom.core/not-found, :pagination/has-prev? :com.wsscode.pathom.core/not-found, :pagination/edges :com.wsscode.pathom.core/not-found} {:pagination/first-cursor :com.wsscode.pathom.core/not-found, :pagination/has-next? :com.wsscode.pathom.core/not-found, :pagination/last-cursor :com.wsscode.pathom.core/not-found, :pagination/has-prev? :com.wsscode.pathom.core/not-found, :pagination/edges :com.wsscode.pathom.core/not-found} {:pagination/first-cursor :com.wsscode.pathom.core/not-found, :pagination/has-next? :com.wsscode.pathom.core/not-found, :pagination/last-cursor :com.wsscode.pathom.core/not-found, :pagination/has-prev? :com.wsscode.pathom.core/not-found, :pagination/edges :com.wsscode.pathom.core/not-found}]
#2020-10-1120:42yendaanother thing that bothers me is that I add to set the ::p/union-path globally
::p/union-path
                     (fn [env]
                       (let [e (p/entity env)]
                         (:notification/type e)))
I was hoping I could limit it to the resolver
#2020-10-1210:56pithylessIs there some way of passing additional data with ::p/continue? I'd like to have several resolvers with different authentication strategies try to resolve the same attribute (and ::p/continue if they fail); and ideally I'd like to differentiate between ::p/not-found and ::auth/forbidden if they all fail. If this is currently not feasible, is this a use-case that can be considered for the upcoming new reader?#2020-10-1218:38wilkerluciomaybe using error instead of continue, similar to "continue", an error will trigger other options, and in a post-process you can convert those errors in forbidden, makes sense?#2020-10-1300:25pithylessYeah, make sense; an authentication error could be considered the same class of problems as an arbitrary error thrown by a resolver. This can then be post-processed in multiple ways (show all errors, silently drop, etc). I guess the only wrinkle is you can't return a partial result (e.g. some connect attributes are OK, others forbidden), so you need separate resolvers for attributes that may be forbidden. That's probably not a bad trade-off to avoid complicating the code paths. šŸ™‚ Thanks!#2020-10-1302:36wilkerlucioI think this is something interesting to consider about error handling in pathom 3, in the current only the last error remains, but I think could be more interesting to see all that happened, not just the last error#2020-10-1219:05wilkerluciohello everyone, I wrote a post about some of the updates about Pathom 3, you can find it at: https://blog.wsscode.com/pathom-updates-01/#2020-10-1219:08wilkerlucioanother news, I recently released [com.wsscode.pathom "2.3.0-alpha13"] šŸŽ‰, this version ports back the defresolver from Pathom 3 to Pathom 2, this defresolver reduces the boilerplate required on some resolvers, it can make some inputs and outputs implicit, for example:
; before alpha13
(pc/defresolver full-name [env {::keys [first-name last-name]}]
  {::pc/input  #{::first-name ::last-name}
   ::pc/output [::full-name]}
  {::full-name (str first-name " " last-name)})

; after alpha13, this can reduce to
(pc/defresolver full-name [{::keys [first-name last-name]}]
  {::full-name (str first-name " " last-name)})
Check the defresolver docstring for details of the syntax.
#2020-10-1316:24wilkerluciothere was a bug with implicit inputs on alpha13, please update to alpha15 to fix#2020-10-1220:14mpenetNice! I am eager to try v3#2020-10-1314:33wilkerlucio#2020-10-1318:40dehlido implicit inputs/outputs work for resolvers that return core async channels? Edit: Looks like it doesn’t; I’ll dig a little deeper to see what’s going on#2020-10-1319:35wilkerlucioinputs yes, outputs no, its important to take this message with care "only works when the last form of the defresolver body is a map", if you have (let [...] {:map 3}), the last form is a let, not a map, and wont work, implicit outputs are meant for only trivial output cases, the async problem is the same as let, the map is not the last form#2020-10-1319:36wilkerlucioit may support a bit more in the future, like let, but I'm holding on it, because in other cases like cond, if, etc... the answer is not as simple#2020-10-1319:45dehliThanks @U066U8JQJ that makes sense to me!#2020-10-1319:56dehliI’m thinking of having an async resolver that just returns a top-level key another resolver that takes that as input and can fully utilize implicit outputs. Something like:
(pc/defresolver async-resolver
  [{:keys [db-get]}
   {:resource/keys [uuid]}]
  {::pc/outputs [:resource/db-item]}
  (go
    {:resource/db-item 
     (<! (db-get uuid))}))

(pc/defresolver sync-resolver
  [{:resource/keys [db-item]}]
  {:resource/foo (:foo db-item)
   :resource/bar (:bar db-item)})
Is there some better way to chain these together?
#2020-10-1319:58wilkerlucioI suggest don't going further just to avoid writing the output, think more like this: if I can get implicit outputs, great, but if not, write by hand, it doens't worth optimize for it#2020-10-1414:25njjIs there any way to get a more descriptive error of what went wrong other than ā€œreader errorā€?#2020-10-1415:12dehliI’m not sure if this will address your issue but you can add a custom error processor which you can use to pull out additional information (https://cljdoc.org/d/com.wsscode/pathom/2.2.30/doc/pathom-developers-guide#_error_handling)#2020-10-1415:31njjThanks I’ll give it a shot#2020-10-1615:50dehliIs there a standardized way to handle access control with pathom? i have an upstream resolver that checks for access but I’d like to prevent downstream resolvers from resolving for cases where access is restricted. I can do a runtime check in each resolver to see if the upstream resolved value is something like :access-denied but I’m curious if there’s something more baked into pathom (I couldn’t find anything when looking through the docs)#2020-10-1615:57souenzzohttps://github.com/souenzzo/eql-style-guide/issues/4#2020-10-1615:57dehlithanks!#2020-10-1615:59souenzzoshort answer: no, but we can build one šŸ™‚#2020-10-1616:01dehliya, that discussion helps a lot! appreciate it!#2020-10-1618:45dehliWhen I pass identity as {::pc/transform identity} to my defresolver I’m getting identity - failed: fn? Has anyone seen something similar? (I’m using alpha15) Edit: I think it’s a bug. I’ll get a test pushed up and see if I can trace it. Here’s the failing test: https://github.com/wilkerlucio/pathom/pull/176 and a potential fix.#2020-10-1823:11wilkerluciothanks, I’ll check on it asap#2020-10-2015:59dehliMy PR didn’t fully address the issue so I went ahead and closed it and will reopen when I get a better solution.#2020-10-2016:01dehliI believe the problem is that the spec is checking at compilation time rather than run time so to the compiler it’s just seeing transform as a symbol, persistentlist, etc.#2020-10-2016:29wilkerluciogotcha, makes sense, I'm doing a few tests here, I guess the solution is to move the validation into resolver instead of defresolver for the options map, this way validate on the realised values (instead of the macro stuff)#2020-10-2016:33wilkerlucio@U2U78HT5G can you try alpha16-SNAPSHOT and see if that works as expected for you?#2020-10-2016:36dehliawesome! i’ll give that a shot!#2020-10-2016:41dehliJust confirmed that the bug is gone šŸ‘ thanks for the help!#2020-10-2016:42wilkerluciothis was the change in case someone is curious: https://github.com/wilkerlucio/pathom/commit/c9292962b54c2ad2820a626a8c82522798fa40bf#2020-10-2016:54dehlithanks! with it now being a runtime check should we wrap it with (when p.misc/INCLUDE_SPECS ?#2020-10-2016:56wilkerlucioI have a little concern with performance, but not much, unless someone is doing resolver generation on the fly all the time, this shouldn't be an issue, and for those cases the user can create the resolver without going though pc/resolver#2020-10-2016:56wilkerluciothe specs on connect are using guardrails to define, they don't check for p.misc/INCLUDE_SPECS#2020-10-2017:11dehligotcha; thanks for the explanation!#2020-10-2017:35wilkerluciono worries, and thanks for updating the issue#2020-10-1703:35zhuxun2Does Pathom's resolving mechanism implies that an n-step walk along the edges would typically result in n accesses to the database? Wouldn't that incur a performance penalty for very deep pulls? Has it ever been an issue?#2020-10-1703:52dehliI haven’t personally used this yet but I remember reading through https://blog.wsscode.com/pathom/v2/pathom/2.2.0/connect/resolvers.html#_n1_queries_and_batch_resolvers which discusses an alternative to the issue you describe#2020-10-1821:12Thomas Moerman@wilkerlucio I added a comment to this (closed) issue, I have the impression that union queries for the parallel parser are still broken, or i'm doing something wrong: https://github.com/wilkerlucio/pathom/issues/160#2020-10-1823:12wilkerluciothanks, I’ll have a look on it tomorrow#2020-10-1909:25Thomas Moermangreat, thanks#2020-10-1910:12yenda@wilkerlucio also related, when trying to use join-seq in a resolver for a union query, I have to use a weird workaround: https://clojurians.slack.com/archives/C87NB2CFN/p1602448785082500#2020-10-1918:56wilkerlucio@U052A8RUT sent a comment there#2020-10-1918:58wilkerlucio@U0DB715GU I understand this a pain at the moment, one idea I have to go around is to make a ::p/union-path that is smarter and can do both (the default and yours, depending on the context), you can use information from env to decide the context, maybe you change depending on the ::p/path, makes sense?#2020-10-1919:20Thomas Moerman@wilkerlucio I answered with an example that triggers the problem#2020-10-1919:24wilkerluciohello, I tried with to many with this:#2020-10-1919:24wilkerlucio
(pc/defresolver input-group=>cases-union
  [_ {:nexus.project/keys     [historic-cases]
      :nexus.input-group/keys [id] :as input}]
  {::pc/docstring "Resolve Cases by InputGroup, union query."
   ::pc/input     #{:nexus.input-group/id
                    :nexus.project/historic-cases}
   ::pc/output    [{:nexus.input-group/cases-union [:nexus.case.image-annotation/id
                                                    :nexus.case.test-case/id]}]}
  {:nexus.input-group/cases-union [{:nexus.case.image-annotation/id 1}
                                   {:nexus.case.test-case/id 1}]})

(def debug-parser
  (p/parallel-parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/parallel-reader
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate-async
     ::p/plugins [(pc/connect-plugin {::pc/register [input-group=>cases-union
                                                     (pc/constantly-resolver :nexus.project/historic-cases "foo")]})
                  p/error-handler-plugin
                  p/trace-plugin]}))

(comment
  (async/<!! (debug-parser {}
     [{[:nexus.input-group/id 1]
       [{:nexus.input-group/cases-union {:nexus.case.image-annotation/id [:nexus.case.image-annotation/id
                                                                          :cmmn.case/id
                                                                          :cmmn.case/name]
                                         :nexus.case.test-case/id        [:nexus.case.test-case/id
                                                                          :cmmn.case/id
                                                                          :cmmn.case/name]}}]}])))
#2020-10-1919:24wilkerlucioit still worked, Im wondering what is triggering the error validation (I'm using with guardrails on here too, but not triggering the error)#2020-10-1920:37njjIs there anyway to avoid full paths to mutations?#2020-10-1920:37njj
(pc/defmutation submit-space-update [env inputs]
  {::pc/sym `app.ui.listing.mutations/submit-space-update}
  (log/info "Submit space" inputs))
#2020-10-1920:37njjit just seems a little overkill for me to have to use the full path in pc/sym to link up to the client side mutations for any server side mutations I have#2020-10-1921:43wilkerlucioare you looking for a shorter way to define or to call the mutation?#2020-10-2011:02njj@U066U8JQJ I’m looking for a way to not have the long full path to the client side mutation#2020-10-2012:44souenzzo@U010LFZGFEG you can use alias (just on /cdn-cgi/l/email-protection) Just like this: https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L22 And this is a "know issue" from clojure. Aliasing without require may be solved at language-level in some future.#2020-10-2012:58njj@U2J4FRT2T so in my case I’d do (alias 'listing 'app.ui.listing.mutations)#2020-10-2012:59njjthen I could do listing/submit-space-update?#2020-10-2022:15wilkerluciothe issue that could fix this problem is https://clojure.atlassian.net/plugins/servlet/mobile?originPath=%2Fbrowse%2FCLJ-2123#issue/CLJ-2123#2020-10-2113:55njjI ended up co-located the clj/cljs mutation files so they have the same namespace in the mean time#2020-10-1920:56nivekuilsyntax quote automatically namespace qualifies symbols#2020-10-2013:11ianjonesHey, could anyone point me to an example on how to wrap a rest api in pathom for a Fulcro app?#2020-10-2013:33souenzzo- Create a transmit! function to "adapt" pathom into "remote" interface https://github.com/souenzzo/eql-realworld-example-app/blob/master/src/conduit/client/rest.cljs#L17#2020-10-2013:34souenzzo- Create a parser with the resolvers from your rest API https://github.com/souenzzo/eql-realworld-example-app/blob/master/src/conduit/client/rest.cljs#L32#2020-10-2013:34souenzzoHere the resolvers example https://github.com/souenzzo/eql-realworld-example-app/blob/master/src/conduit/connect/proxy.cljc#L236#2020-10-2013:54ianjonesthank you! im doing the flux challenge just to wrap my head around fulcro pathom and really hit a road block because most of the examples assume you want graphql or are talking to a db#2020-10-2013:56ianjoneshttps://github.com/staltz/flux-challenge#2020-10-2014:27ianjones@U2J4FRT2T do you have a blog or anything where you write about things? I’d subscribe šŸ‘€#2020-10-2015:02dvingoi have another example hitting the wikipedia api: https://github.com/dvingo/pathom-client-wikipedia/blob/master/src/main/dv/pathom_wp/client/application.cljs#L18 hosted on github pages: https://dvingo.github.io/pathom-client-wikipedia/#2020-10-2015:03dvingouses this helper: https://github.com/dvingo/my-clj-utils/blob/master/src/main/dv/fulcro_util.cljs#L533#2020-10-2101:18ianjonesI got it working! Thanks for the help y’all#2020-10-2015:56yendaCurrently pathom returns only one result for the same mutation, for instance if the user has a connection issue and the query grows into calling the same mutation with different params twice, I only get the result of the last one. Is there a way to have both or could it be included in a future version?#2020-10-2020:36wilkerluciowhat you mean by query grows into calling same mutation with different params? in case of a retry isnt it a different request?#2020-10-2023:16tekacs@U066U8JQJ I think I came here to ask this exact question myself, too šŸ˜†
[(send-message {:message/text "hey"})
 (send-message {:message/text "there"})]

#_=>

{send-message {:message/sent "there"}}
that's just an example, but the gist is that with mutations sitting at the root (and you can't AFAICT join them under placeholder prefixes either), if you send a query that uses (send-message ...) twice, there's not an obvious way to capture the return values from both of those mutations, since they both write into {send-message ...} at the top level. Does that make it clearer?
#2020-10-2023:17tekacsI would do this or something similar if I could:
[{:>/first (send-message ...)}
 {:>/second (send-message ...)}]
but that comes out as an invalid join or maybe something like this, possibly?
[{(send-message ...) {:>/first [:message/sent]}}
 {(send-message ...) {:>/second [:message/sent]}}]
but that just yields an empty result map, like this:
{send-message {}}
#2020-10-2112:51wilkerluciohello, for your case @U0H98U6SY, you can go around this limitation by writing a mutation that sends multiple messages at once#2020-10-2112:51wilkerlucioor, some sort of mutation that composes other mutaitons#2020-10-2114:54yenda@U066U8JQJ I merge queries into one until it can be sent, on retry it does one single query with all the pending ones#2020-10-2117:21tekacsright, what yenda said @U066U8JQJ#2020-10-2117:22tekacsmerging them into one doesn't really help, because it'd break ordering for one thing#2020-10-2117:22tekacs
[(mutation-1)
 (mutation-2)
 (mutation-1)]
#2020-10-2117:22tekacsthe only real option as far as I can tell is to manually split the mutations externally to Pathom and then to send them into the engine one at a time šŸ˜ž#2020-10-2117:28tekacsI guess I could write a mutation that composes other mutations...#2020-10-2117:29tekacsbut would be wonderful if there were an ad-hoc way of simply giving each mutation its output name#2020-10-2117:44yendawhich is why I was wondering if it could return a collection of results when the mutation was called more than once#2020-10-2117:45yendaif I'm not mistaken that would not even break existing queries, since for instance
(defn block-user
  [user-id blocked?]
  {(list 'block-user {:user/id user-id
                      :user/blocked? blocked?})
   [:user/id
    :user/blocked?]})
could anyway receive either one map or a coll of map
#2020-10-2118:01tekacsif we could rename outputs, then we could get the same effect as a collection by transforming queries to/from a form with renames#2020-10-2118:01tekacsthat's why that'd be my aim#2020-10-2118:01tekacsI might see if I can do that in a plugin#2020-10-2118:04yendaIf I'm not mistaken the overwrite occurs in mutate and mutate-async when the result is merged in the env#2020-10-2118:16yendamhm actually looks like it's in the parser#2020-10-2118:17yenda
(recur (assoc res (ast->out-key ast) value) tail)
#2020-10-2118:21yendawould it be acceptable to do
(let [out-key (ast->out-key ast)
      previous-res? (get res out-key)]
  (recur (if previous-res?
           (update res out-key (fn [previous-res]
                                 (if (vector? previous-res)
                                   (conj previous-res res)
                                   [previous-res value])))
           (assoc res out-key value)) tail))
#2020-10-2118:44wilkerlucio@U0DB715GU the vector thing is a problem, because it breaks the assumption that mutations always return a map, I think a behavior like that could be confusing. there is a plugin entry point that I've been considering for a while, and I think it could solve this problem, is a ::p/wrap-output, something to run exactly when pathom is building the output, with this hook you could either write a plugin to use some param and have some different output name (like alias works for reads today) or even make what you are suggesting. this was planned for Pathom 3, but I can make in Pathom 2, this can also give some room to experiment with this entry point#2020-10-2118:53yendayeah the plugin way would be satisfying enough, I had the assumption mutations would return a vector when ran more than once like resolvers when they return more than one result šŸ˜„#2020-10-2018:30yendaIt could return a vector of results in the mutation key?#2020-10-2119:11tekacsI would curious to hear if there any plans to add Pathom 3 defresolver-style sugaring (inference of params, output, removing env/params) to the defmutation syntax, at all?#2020-10-2119:35wilkerluciomaybe, I didn't started mutations in Pathom 3 yet#2020-10-2119:48wilkerluciobut I think it should, just instead of inferring inputs, it will infer params#2020-10-2218:40tvaughanI have a parallel parser that is called in some http-kit middleware (for server-side rendering). This has been working for quite some time without issue, but just now broke when I added additional data to a test fixture. I think what's happening is that the database query takes just long enough now (despite being extremely quick) for http-kit to trigger some sort of timeout. This is pure speculation, but reducing the size of the test fixture and switching to a normal parser both solve the problem. Does this ring a bell for anyone? What am I doing wrong? Below is what I had. drawings-endpoint is the function called in the http-kit middleware.
(defonce ^:private drawings-parser
  (pathom/parallel-parser
    {::pathom/env {::pathom/reader [pathom/map-reader
                                    pathom-connect/parallel-reader
                                    pathom-connect/open-ident-reader
                                    pathom-connect/index-reader]
                   ::pathom/process-error ex-handler
                   ::pathom-connect/mutation-join-globals [:tempids]}
     ::pathom/mutate pathom-connect/mutate-async
     ::pathom/plugins [(pathom-connect/connect-plugin {::pathom-connect/register drawings-handlers})
                       pathom/error-handler-plugin
                       pathom/trace-plugin]}))

(defn drawings-endpoint
  ([query]
   (drawings-endpoint query {}))
  ([query opts]
   (when query
     (<!! (drawings-parser opts query)))))
And this my "solution"
(defonce ^:private drawings-parser
-  (pathom/parallel-parser
+  (pathom/parser
     {::pathom/env {::pathom/reader [pathom/map-reader
-                                    pathom-connect/parallel-reader
+                                    pathom-connect/reader2
                                     pathom-connect/open-ident-reader
                                     pathom-connect/index-reader]
                    ::pathom/process-error ex-handler
                    ::pathom-connect/mutation-join-globals [:tempids]}
-     ::pathom/mutate pathom-connect/mutate-async
+     ::pathom/mutate pathom-connect/mutate
      ::pathom/plugins [(pathom-connect/connect-plugin {::pathom-connect/register drawings-handlers})
                        pathom/error-handler-plugin
                        pathom/trace-plugin]}))
@@ -32,6 +32,7 @@
   ([query opts]
    (when query
-     (<!! (drawings-parser opts query)))))
+     (drawings-parser opts query))))
 
#2020-10-2219:10souenzzo@tvaughan looks like that you are doing Locking IO inside parallel-parsers resolvers You can solve it by using a thread-pool or use a non-blocking db api that probablly will have a internal thread-pool In real, looks like that your use-case a simple "serial" parser will be a better solution#2020-10-2219:12tvaughanThe db query is a simple datomic pull-syntax read operation#2020-10-2219:12souenzzoIf you are using #datomic cloud I highly recommend use p/parser I started with a parallel-parser, and after A LOT of debbuging, thread pool, and others patterns/tweaks, to make datomic.client.(async).api work with parallel-parser, I see that p/parser is more performatic šŸ˜ž#2020-10-2219:13tvaughanI switched to the parallel parser because it seemed like (somewhat unconfirmed) that using the serial parser would block all other pending requests#2020-10-2219:14tvaughan> I see thatĀ `p/parser`Ā is more performatic Interesting. I came to the opposite conclusion#2020-10-2219:21souenzzoI started with that conclusion too, for small/devlocal dataset But for larger data, running inside ion, with "hot" cache.. looks like that datomic already do the threading internally for you. Many threads accessing datomic api just cause a "thread overhead" PS: my conclusion is about datomic IONS If you are using datomic-cloud without ions, that relay on http calls, you may end up into different results.#2020-10-2219:34tvaughanCool. Thanks for this#2020-10-2219:14tvaughanThanks @souenzzo I'll stick with the serial parser#2020-10-2219:21souenzzoI started with that conclusion too, for small/devlocal dataset But for larger data, running inside ion, with "hot" cache.. looks like that datomic already do the threading internally for you. Many threads accessing datomic api just cause a "thread overhead" PS: my conclusion is about datomic IONS If you are using datomic-cloud without ions, that relay on http calls, you may end up into different results.#2020-10-2219:23tvaughanCurrently on-prem, but cloud eventually#2020-10-2219:24wilkerlucio@tvaughan serial parser is better for most users, parallel parser is way too complex and adds a ton of overhead, to cross the "overhead gap" you must have quite large queries (and I mean 300+ attributes in a single query) and resolvers that have IO that is easy to distribute (for example hitting many different services)#2020-10-2219:24wilkerluciothe serial is not going to prevent many requests at once, the serial means that for one EQL query, that query is processed sequentially (one attribute at a time)#2020-10-2219:25wilkerluciowhile the parallel can parallelize differnet attributes on the same query#2020-10-2219:26wilkerluciobut I can undestand the confusion, for a time I used to recommand the parallel as the starter, but time goes on and we learn better šŸ™‚#2020-10-2219:27tvaughanTo be clear, the parser in question is used both for server side rendering and as an api endpoint. The performance characteristics I mention are related to its use as an api endpoint. The problem I mention above shows up when used in http-kit middelware. I was surprised that parallel requests to http-kit would block. However, I'm not so surprised that using a parallel reader in http-kit middeware is a problem#2020-10-2219:28tvaughanCool. Thanks for the clarification @wilkerlucio. That does clarify its purpose for me#2020-10-2219:30tvaughan> for a time I used to recommand the parallel as the starter, First I went back to the documentation. FYI - https://blog.wsscode.com/pathom/v2/pathom/2.2.0/core/async.html says: > Nowadays the parallel parser is the recommended one to use ...#2020-10-2219:31wilkerluciothanks, bad old docs, going to update this now šŸ™‚#2020-10-2219:54wilkerluciothank you for the nudge! https://github.com/wilkerlucio/pathom/commit/9827d1dd671cfbd9ed7521a7ca3f29de7e00ed5c#2020-10-2220:02tvaughanThat's super helpful. Thanks!#2020-10-2219:44nivekuilsince pathom3 will only have reader3, how will async resolvers work? You can't return core.async channels from resolvers/mutations anymore right?#2020-10-2219:54dehliI’m using reader3 with core.async channels and it’s working for me#2020-10-2219:56wilkerluciorest assured that Pathom 3 will support async šŸ™‚ in Pathom 2 reader3 supports sync and async. When I say pathom 3 will only have reader3, is more of a way to say the type of processing it will do, because Pathom 3 will not have readers at all (neither parsers).#2020-10-2220:06nivekuilIn reply to @￱￱slack￱￱￱￱￱￱t03￱￱r￱￱z￱￱g￱￱p￱￱f￱￱r=2d￱￱u2￱￱u78￱￱h￱￱t5ļæ±_ļæ±g:nivekuil.comI’m using reader3 with core.async channels and it’s working for meah, it does -- I thought you had to use the parallel reader alongside the parallel parser/mutate. I guess that confusion can't happen in pathom3 if it's not even a thing anymore :)#2020-10-2220:29nivekuilso just changing from parallel-reader to reader3, still using parallel-parser, seems to cause a few random attributes (like 1% of the stuff returned from a big query) that my fulcro app previously loaded fine to be nil. Is this expected?#2020-10-2220:42wilkerlucioreader3 is experimental, so errors are expected yes#2020-10-2300:11souenzzonot sure what expect from mixed parallel-parser with reader3 About reader3, I know some issues if you use placeholders and (:ast env)#2020-10-2419:26lilactownnot sure if this is the right place, but the EQL cljdocs site is broken atm#2020-10-2421:42souenzzohttps://clojurians.slack.com/archives/C8V0BQ0M6/p1603575742106400#2020-10-2514:12wilkerluciofixed: https://cljdoc.org/d/edn-query-language/eql/1.0.1#2020-10-2514:12wilkerluciothanks, reporting here works, there is also #eql#2020-10-2422:50dehliHi, I have a mutation that I’d like to take both ::pc/input and ::pc/params (where I’d like the ::pc/input the be an attribute that’s globally resolvable). I’ve tried to get this to work but the input isn’t resolving. Is this supported? Thanks!
(pc/defmutation send-login-email
  [{:keys [send-email]} {:app/keys [origin] :login/keys [email]}]
  {::pc/sym 'login/send-email
   ::pc/input #{:app/origin}
   ::pc/params [:login/email]
   ::pc/output [:login/sent]}
  (go
    (<! (send-email {:email email :origin origin}))
    {:login/sent true}))
#2020-10-2501:31cjmurphyinputs are not supported for mutations as they are for resolvers. šŸ˜ž#2020-10-2501:32souenzzoJust call the parser from the mutation
(pc/defmutation send-login-email
  [{:keys [parser] :as env} {:app/keys [origin] :login/keys [email]}]
  {::pc/sym 'login/send-email
   ::pc/params [:login/email]
   ::pc/output [:login/sent]}
  (go
    (<! (send-email {:email email :origin (:app/origin (<! (parser env [:app/origin])}))))
    {:login/sent true}))
#2020-10-2502:20dehliOoohhh, that's awesome! Didnt realize I could do that. Thanks!#2020-10-2623:20wilkerlucioPathom Updates #2 is out! https://blog.wsscode.com/pathom-updates-02/#2020-10-2712:02souenzzoAnything about the new placeholder//dataholders?#2020-10-2712:20wilkerlucionot at this time#2020-10-2703:47lilactownare there any libraries that implement querying regular core data (maps/vecs/sets/etc.) using the EQL spec?#2020-10-2710:58wilkerluciohttps://cljdoc.org/d/com.wsscode/pathom/2.3.0-alpha16/api/com.wsscode.pathom.core#map-select#2020-10-2710:58wilkerluciopathom has this utility#2020-10-2712:02souenzzoNot strictly EQL https://github.com/juxt/pull#2020-10-2715:26lilactowngreat thanks!#2020-10-2916:55Ahmed Hassan> default > namespace What is that?#2020-10-2917:10souenzzo@UCMNZLJ93 https://blog.wsscode.com/pathom/v2/pathom/2.2.0/core/placeholders.html#2020-10-2703:48lilactown(if not I might look at releasing some code I just wrote)#2020-10-2811:46mpenetin case you missed it: https://soundcloud.com/user-959992602/s4-e15-pathom-with-wilker-lucio-part-1#2020-10-2910:46Thomas MoermanGuys, opinion question: In my current project we use Pathom with Crux db. Now, as there is no "ORM" layer of sorts, and some subsystems (non-UI) require complex data resolution, it appears to me that you could use Pathom as well in that context instead of only as a means to feed the UI front-end. What do you guys think of this? Suggestions?#2020-10-2912:08souenzzoHello @U052A8RUT I see #pathom as something like a "ORM" inside clojure way to program It's stable layer there I can ask "give me the user/address" and it will get in DB, FancyDB, Cache, HTTP, I can chage the DB schema, store data in a new format and still provide the same user/address. I use pathom to manage REST API's and SSR pages at my work. I "export" my endpoints with something like that: https://github.com/souenzzo/eql-as#real-world-exmaple#2020-10-2912:09Thomas MoermanThanks, i'll have a look soon šŸ‘#2020-10-2914:56wilkerlucio@U052A8RUT, yes, I don't see pathom as a API thing, but as a way to do "attribute modeling", which is the idea to create data processing mechanisms based on the attribute relationships#2020-10-2914:56wilkerluciothat wasn't what Pathom was made intiially for, but its where its landing, the new docs reflect this vision better, hope to get that out soon#2020-10-2914:59Thomas MoermanI agree completely, I have the feeling that Pathom implements such a fundamental idea that we are still "discovering" ways in which we can use it.#2020-10-2915:01Thomas MoermanIt sometimes reminds me of the Star Trek transporter, in need of some piece of data? -> energize! there it is! šŸ˜‰#2020-11-0121:10Ahmed Hassanhttps://jeffhandley.com/2018-09-13/graphql-is-not-odata#2020-11-0121:10Ahmed Hassanhttps://jeffhandley.com/2018-09-13/graphql-is-not-odata#2020-10-3019:18bedershello there, quick question: Are parameters in Pathom EQL supported? I’m using this query: [{(:account/all {:search "bubu"}) [:account/name]}] but are getting an ā€˜Invalid expression’ exception#2020-10-3019:19bedersthe example in the manual about Parameters doesn’t use joins#2020-10-3019:20beders(this is on 2.3.0-alpha16)#2020-10-3019:23wilkerlucio@beders a common pitfall is forgetting to quote the list, otherwise instead of sending it, you make Clojure evaluate, so in terms of Clojure code, you need to write as: [{'(:account/all {:search "bubu"}) [:account/name]}]#2020-10-3019:24bedersoh, jesus, you are right šŸ™‚#2020-10-3019:24bedersthanks for the quick response!#2020-10-3019:24bedersI’m getting blind to list expressions#2020-10-3019:25bedersour team of 3 is discovering the wonders of pathom and love it!#2020-10-3122:59lilactowntrying to understand the EQL spec a bit better w.r.t. recursion. is the expectation for a query like:
[{[:entry/name "foo"] [:entry/name {:entry/folders ...}]}]
which results in this AST:
{:type :root,
 :children
 [{:type :join,
   :dispatch-key :entry/name,
   :key [:entry/name "foo"],
   :query [:entry/name #:entry{:folders ...}],
   :children
   [{:type :prop, :dispatch-key :entry/name, :key :entry/name}
    {:type :join,
     :dispatch-key :entry/folders,
     :key :entry/folders,
     :query ...}]}]}
that when I am interpreting the :entry/folders join, I will repeat the previous query [:entry/name #:entry{:folders ...}] ?
#2020-11-0101:08wilkerlucioyes, that's correct#2020-11-0101:09wilkerluciofor reference, this is where the pathom parser does it: https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/parser.cljc#L236#2020-11-0123:34bbssThe planner debugger looks incredibly powerful. Will this be available for clj or just cljs? Is there a way I can kick the tires?#2020-11-0201:22wilkerluciowill be available for both, not public yet#2020-11-0201:22wilkerluciobut just for the fun, I did some changes, for the curious ones, this is what its looking now:#2020-11-0202:57souenzzoHello šŸŽƒ I did a small prototype of a possible connector between #re-frame and #pathom I would like some feedback of any kind There is 3 main points: - A reg-fx that simply call the parser. it receive a eql query and return's a result - A tree->db function, that ✨ smartly ✨ merge the db with the result, using the query and it's metadata to do the normalization stuff - A db->tree function, that given a query, it denormalize the DB The data fetched will be normalized as you describe in the query. The final DB can be like a #fulcro db, where you want, or a regular "deep tree db" if you prefer. You will not need to create tons of reg-sub or reg-event, but you can create or migrate slowly as you need. https://github.com/souenzzo/souenzzo.github.io/blob/master/eql-refdb/dev/sample/app.cljs#2020-11-0213:56tvaughanI need to restrict queries based on the current user, so I want to write a plugin that is able to provide a map about the current user to the queries using the ring session data. I don't want to trust anything the client sends about who the current user is other than trusting the session token. Is a plugin the right approach? Are there any examples out there on how to do this? Thanks#2020-11-0214:05tvaughanTo add, we're using Fulcro so http://book.fulcrologic.com/#_example_securing_a_normal_remote is an option. However, I'm still unsure how to pass additional data that isn't part of the client query to Pathom#2020-11-0214:06souenzzoThere is some topics here https://github.com/souenzzo/eql-style-guide/issues/4 Any standard/final solution AFIK#2020-11-0214:14tvaughanAh ok. I did not think to look at eql-style-guide. Thanks#2020-11-0214:06eoliphantwe’ve had to do something similar. We convert the token into a user id in the middleware and jam the user id into the env when the parser executes. datomic is our primary backend datastore, and we have some rules that limit the results based on the user. For mutations we just check the perms with a wrapper around the ā€˜real’ function. I messed around with a plugin approach but it was hard to make it work ā€˜generically’#2020-11-0214:12tvaughan> and jam the user id into the env This is the part I haven't figured out yet. Is there an example you could share?#2020-11-0214:12eoliphantsure one sec#2020-11-0214:21eoliphantI’ve pasted in some bits from one of my projects ( this assumes the typical project layout)
; in x.x.server-components.middleware.clj
(defn wrap-api [handler uri]
  (fn [request]
    (if (= uri (:uri request))
      (handle-api-request
        (:transit-params request)
        ; we do the whole request, but you could say just pull the auth header here 
        (fn [tx] (parser {:ring/request request} tx)))
      (handler request))))
; in .x.x.server-components.pathom
;; we wrap this func with core.memoize
(defn userinfo
  "get the userinfo from auth0"
  [env]
  (let [req (:ring/request env)]
    (log/debug :req req)
    (when-let [token (get-in req [:headers "auth-token"])]
      (log/debug "got token" token)
      (log/spy :debug (get-userinfo token)))))
;; in your parser setup
...
::p/plugins [(p/env-wrap-plugin (fn [env]
                (assoc env :userinfo (userinfo env)) 

                       ]
...
#2020-11-0214:27eoliphantI pulled out superfluous stuff that we use. but basically use p/wrap-env-plugin to jack what you’d like into the env#2020-11-0214:42tvaughanNice! Thanks a lot @U380J7PAQ! šŸ”„#2020-11-0214:42eoliphantnp šŸ™‚#2020-11-0214:46eoliphantit’s nice bc our specifics (e.g. Auth0) don’t leak any further, and our users have ā€˜our’ internal UUID and an external-id attribute that we map to what we get back from auth0 (e.g. auth0|xxxxxx), such that we can have multiple providers or switch from one to another#2020-11-0214:50tvaughanCool cool cool. We've taken a similar approach with ids. I'm curious, why did you choose Auth0? I assume your app doesn't have any concept of groups of users or sharing between users, right?#2020-11-0214:55eoliphantwe’d already used it for some other stuff. and we really only use it for identity. Authorization, ā€˜relationships’, etc are managed in the app#2020-11-0321:57lilactownI have a data modeling question. let's say I have a bunch of entities, pets and owners. a pet has an owner, and owners have pets. I have some pathom resolvers:
(pc/defresolver pet-resolver
  [{:keys [db]} {:keys [:pet/id]}]
  {::pc/input #{:pet/id}
   ::pc/output [{:pet [:pet/id :pet/name :pet/age ,,,]}]
  (entity db [:pet/id id]))

(pc/defresolver owner-resolver
  [{:keys [db]} {:keys [:owner/id]}]
  {::pc/input #{:owner/id}
   ::pc/output [{:owner [:owner/id :owner/name :owner/age ,,,]}]
  (entity d/b [:owner/id id]))
now, my idea was that: wouldn't it be easy if each pet just had an :owner/id on their entity in our system of record? this would allow to easily nest the :owner resolver within a query for a pet:
(parse [{[:pet/id 123] [:pet/id :pet/name {:owner [:owner/id :owner/name]}]}])
;; => {[:pet/id 123] {:pet/id 123 :pet/name "Spot" :owner {:owner/id 9 :owner/name "lilactown"}}}
#2020-11-0321:58lilactownhowever, this introduces a problem in my system of record: if I'm using something like datomic/datascript, suddenly I have entities that have a key :owner/id, which are not in fact owners. :owner/id no longer specifies a unique identity of an entity#2020-11-0414:14wilkerluciothat property is fine, as long as the children entities only link to one owner#2020-11-0322:05jmayaalvmaybe :pet/owner-id and have an alias resolver to :owner/id ?#2020-11-0322:05lilactownthis could lead me to changing this in a couple of different ways. I could have a :pet/owner-id key and a special pet-owner-resolver which takes that key and looks up the owner.
(pc/defresolver pet-owner-resolver
  [{:keys [db] {:keys [:pet/owner-id]}]
  {::pc/input #{:pet/owner-id}
   ::pc/output [,,,]}
  (entity db [:owner/id owner-id]))
However this is a lot of repetitive code, and my system actually has many different kinds of entities all of which can potentially have M:N relationships with each other. another option is to nest the reference: I could have a :pet/owner key which contains a map {:owner/id 9} in my system of record. this way, the key isn't on an entity but rather it will be interpreted as a ref by datomic/datascript. This changes the query a bit:
(parse [{[:pet/id 123] [:pet/id :pet/name {:pet/owner [{:owner [:owner/id :owner/name]}]}]}])
;; => {[:pet/id 123] {:pet/id 123 :pet/name "Spot" :pet/owner {:owner {:owner/id 9 :owner/name "lilactown}}}}
This additional nesting... annoying? Tedious? It feels like wasted complexity
#2020-11-0322:06lilactown> maybeĀ `:pet/owner-id`Ā Ā  and have an alias resolver toĀ `:owner/id`Ā ? that probably is the answer. I guess there are utilities for creating these that remove the boilerplate?#2020-11-0322:08jmayaalvyes, it’s easy šŸ™‚ you can create an a single direction alias or a bidirectional alias https://blog.wsscode.com/pathom/v2/pathom/2.2.0/connect/resolvers.html#2020-11-0411:26souenzzo@lilactown I would like this:
(pc/defresolver pet-pull
  [_ _]
  {::pc/input #{:pet/id}
   ::pc/output [:pet/id :pet/name :pet/age ,,,]
  ...)
(pc/defresolver owner-pull
  [_ _]
  {::pc/input #{:owner/id}
   ::pc/output [:owner/id :owner/name :owner/age ,,,]}
  ...)
(pc/defresolver pet-owner
  [_ _]
  {::pc/input #{:pet/id}
   ::pc/output [:owner/id]}
  ...)

(pc/defresolver owner-pets
  [_ _]
  {::pc/input #{:owner/id}
   ::pc/output [{:owner/pets [:pet/id]}]}
  ...)

[{[:pet/id 123] 
  [:pet/id 
   :pet/name 
   {:>/owner [:owner/id
              :owner/name]}]}]
#2020-11-0414:12wilkerlucio@lilactown I would go close with @souenzzo suggestion, when you wrap the result in things like [{:pet [:pet/id :pet/name :pet/age ,,,]}] you miss out on some connection leverage between the attributes, because of the nesting#2020-11-0414:15wilkerluciobut the alias is also a good option, and keeps the data easier to reason (you can see the alias going on)#2020-11-0519:36markaddlemanI'm reading through Pathom3 docs. First, congratulations on excellent tutorial and documentation! The built-in resolvers look very convenient. Have you considered a built-in Parameters resolver? I imagine it would be similar to the Environment resolver but examine [:ast :params]#2020-11-0613:46wilkerluciothanks šŸ™‚ do you have a use case in mind for the params resolver generator?#2020-11-0614:36markaddlemanBear in mind that I'm pretty new to Pathom2 and haven't explored Pathom3 beyond reading the docs. Here is what I was thinking: With parameter support, the smart map becomes an immutable, central computing object. I would assoc in parameters and ask for a result back - and it would do all of the hard work of figuring out how to string together functions (resolvers) to obtain the result. From a more mundane point of view, I have several resolvers that require parameters. It is inconvenient (and frankly a little weird) to inspect (-> env :ast :params) to obtain the information. Instead, it would be much more convenient to declare parameters as inputs to the resolver.#2020-11-0614:38markaddlemanCarrying this last example just a little further, I guess that most resolvers that require the env parameter only require it in order to access parameters. With a Parameter resolver, I imagine nearly all resolvers become a single arg function. I don't have a firm reason to think this is a good thing but it feels like a move in the right direction.#2020-11-0614:40wilkerlucioat this moment I don't really see much usage on that sense, with any of the helpers you can already use params, maybe there is some misconception going on in our little chat here#2020-11-0614:41wilkerlucioif you have a concrete use case, we can discuss around it#2020-11-0614:42markaddlemanSounds good#2020-11-0620:41ianjones@wilkerlucio I listened to the first part of the clojurescript podcast you are on and its excellent. I enjoyed how you compare how you would document the domain model in REST vs Graphql vs Pathom. It made the differences quite clear to me#2020-11-0620:44ianjoneshttps://soundcloud.com/user-959992602/s4-e15-pathom-with-wilker-lucio-part-1#2020-11-0620:44ianjonesfor those interested#2020-11-0620:45ianjonesI just noticed your RoamResearch db in the pinned section. Thats a super cool way of documenting your project šŸ”„#2020-11-0622:36wilkerluciothanks, I like a lot to use that for daily notes šŸ™‚#2020-11-1112:52dehliHi, I have a question about resolvers and input. The following code doesn’t work, but I’m wondering if there’s a way to accomplish the same thing (or maybe I’m doing something wrong). Also thanks for all the advice so far! I’m really enjoying pathom.
(pc/defresolver authd-resolver
  [_ _]
  ;; This is a globally accessible resolver
  {:authd/user {:user/id "foo"})

(pc/defresolver user-resolver
  [_ {:user/keys [id]}]
  {::pc/output [:user/email]}
  ;; Retrieve the user from db (currently just a stub)
  {:user/email (str id "@bar.com")})

(pc/defresolver custom-resolver
  [_ params]
  ;; this part doesn't work for me and neither does #{:authd/user :user/email} 
  {::pc/input #{{:authd/user [:user/email]}}}
  )
Note: if I simply query for data like below I do get the expected response:
[{:authd/user [:user/email]}]
;; -> 
{:authd/user {:user/email "
#2020-11-1113:41souenzzo@dehli ::pc/input is a (s/coll-of keyword? :kind set?), not a EQL pattern I already requested this feature some time ago. But it's a pretty hard problem. You can use
(pc/defresolver custom-resolver
  [{:keys [parser] :as env} _]
  {::pc/output [...]}
  (let [email (->> [{:authed/user [:user/email]}]
                   (parser env)
                   :authed/user
                   :user/email)]
    ...email...))
#2020-11-1113:42souenzzoIn my current app i created :app.authed-user/email, :app.authed-user/id so i can easily access it.#2020-11-1113:46dehliThanks @souenzzo! I will search for that feature to give it a šŸ‘ but I can imagine it being difficult. I’ll go down the route you suggested of creating a separate resolver for those specific authd fields I need often thought. Thanks again for the help!#2020-11-1113:48souenzzoNot sure if there is a issue for it But in pathom3** the :input is already specified as EQL pattern, although AFIK it still just use the "top level" keys. not released // in design#2020-11-1113:49dehliawesome! pathom3 sounds great šŸ™‚#2020-11-1114:09wilkerlucioyes, I plan to support that in pathom3 in the future, I call ā€œnested inputsā€. this is specially needed when integrating with dynamic resolvers, like graphql#2020-11-1114:09wilkerlucioand like @souenzzo said, currently you can kind achive that by doing a recursive call to the parser#2020-11-1114:51wilkerlucioon difference though, I suggest having an input (without the nesting) in the resolver config anyway, otherwise you risk pathom trying to trigger that resolver even when there is no path available for the input#2020-11-1115:26dehlithanks for the suggestion!#2020-11-1114:50wilkerlucioPathom 3 documentation site is out! That and a few more updates at https://blog.wsscode.com/pathom-updates-03/#2020-11-1120:00adamfeldmanI’m enjoying the second Pathom episode! https://soundcloud.com/user-959992602/s4-e16-pathom-with-wilker-lucio-part-2#2020-11-1211:32thhellerhey I was just going over the pathom3 docs and noticed that env is quite often missing from the examples. I don't think its actually optional right? https://pathom3.wsscode.com/docs/resolvers#2020-11-1211:53pithyless@thheller it looks like a multi-arity function in pathom3; with one arg, env is {} ; with zero args both env and input default to {}#2020-11-1213:29wilkerlucio@thheller @pithyless this is also ported on latest versions of pathom 2, the multi arity thing for defresolver#2020-11-1214:41pithyless@wilkerlucio I know it's rude to ask for ETAs, but are mutations for pathom3 on the roadmap short-term or long-term? If it's not planned for the short-term, I may just try playing with a hybrid approach (trying out P3 resolvers and smart-maps and P2 mutations).#2020-11-1219:32wilkerluciohello, no worries, my rough estimation is to have something that I consider ā€œfeatured enoughā€ to use by the end of this year, and by this I mean reads, mutations, placeholders, batching, telemetry and tooling. prioritization is always tricky, so I may push async later to have something sync funcional first#2020-11-1220:21pithylessThanks for the clarification; looking forward to the improvements in P3! PS. @wilkerlucio Kudos for all the effort you're putting into EQL with you're writing/tech talks/podcasts/etc; it's important long-term work to get more people thinking critically about graphs and data-modeling.#2020-11-1217:21souenzzo@pithyless why move into p3? I'm using p2 with all new features backported from p3 p3 still miss things like placeholders, which it my main "blocking" feature#2020-11-1217:54pithylessI'm interested in some of the new user APIs; like the smart-maps interface and lack of explicit parsers. And I'd like to see if I can re-use some of the graph introspection logic for a slightly different use-case (which probably would be easier to base off of the new internals). But didn't know about placeholders; I guess that's enough reason to keep using p2 for existing code and only use p3 for the more experimental stuff. Thanks for the heads up!#2020-11-1218:09souenzzo
(let [parser (p/parser {::p/plugins [(pc/connect-plugin)
                                     p/error-handler-plugin
                                     p/elide-special-outputs-plugin]
                        ::p/env     {::p/reader               [p/map-reader
                                                               pc/reader2
                                                               pc/open-ident-reader
                                                               p/env-placeholder-reader]
                                     ::p/placeholder-prefixes #{">"}}})
      process (fn [index tx]
                (parser (assoc index
                          ::pc/indexes index)
                        tx))
      index-a-42 (pc/register {} (pc/constantly-resolver :a 42))
      index-dyn-a (pc/register {} (pc/resolver 'a {::pc/output [:a]}
                                               (fn [{:keys [a]} _]
                                                 {:a a})))]
  {"from ctx" (process (assoc index-dyn-a
                         :a 55)
                       [:a])
   "static"   (process index-a-42
                       [:a])})
=> {"from ctx" {:a 55}, "static" {:a 42}}
It's undocumented, but you can decouple indexes from parser in pathom2 šŸ˜‰
#2020-11-1306:03nivekuilmaybe a doc error: under wrap-mutate, I think`{:action out}` should be (:action out)? https://blog.wsscode.com/pathom/v2/pathom/2.2.0/plugins.html#_plugin_skeleton#2020-11-1407:44wilkerluciono, that's correct, the mutation action fn need to be wrapped by the map having the :action key#2020-11-1407:48nivekuilhuh.. I extract the :action key rather than wrapping the map and my plugin works.. same thing as https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/profile.cljc#L46 ?#2020-11-1407:56wilkerlucioah, sorry, you are right, that's just the condition to check if there is an action, so it should be (:action out) šŸ‘#2020-11-1407:56wilkerluciodo you wanna send a PR with the fix?#2020-11-1408:03nivekuilsure, sent šŸ™‚ https://github.com/wilkerlucio/pathom/compare/master...nivekuil:patch-1#2020-11-1408:05nivekuilaw, I've been typing so much fulcro/pathom that I instinctively typed in "env" instead of "out" when I wanted a three-letter variable. can't figure out how to amend the commit message through github#2020-11-1408:08wilkerluciono worries, and site updated#2020-11-1318:34ianjones@wilkerlucio have you release the project you talked about on clojurescript podcast? the one where you are grabbing data from local real estate sites#2020-11-1407:45wilkerlucionot yet#2020-11-1407:46wilkerlucioupdate on the Pathom 3 docs site: now there is search available šŸ˜„ https://pathom3.wsscode.com/#2020-11-1508:04bbssBeen going through the pathom3 docs and examples, the smart map and syntax are really nice! It feels like pathom got simpler with pathom3 and the docs are really good. I have a case that I would like to test out and not sure where to start. Let’s say I have an eql query and a bunch of entities’ attributes will be resolved by a remote api (translation api in this case). The translation api allows batching of requests, so ideally the resolver would wait with the request until it has all the attributes that need to be resolved. I’m thinking I will need to learn about the runner and planner to do something as advanced as this. But don’t see much mention of this kind of parallel/async stuff (yet?). Any pointers or ā€œthis isn’t ready yet/possibleā€ would be appreciated :)#2020-11-1514:48wilkerluciohello, thanks for checking it out, remember this still a in-dev library, currently neighter batching or async are implemented. batching (like pathom 2 has) should come sooner, async/parallel will be done, they are closer to the end of the list#2020-11-1523:52bbssUnderstood, looking good so far šŸ™‚#2020-11-1614:07henrikWhat could be the reason that I get an exception when running this?
(pathom-parser {} [(:instruments {:sort :instrument/type})])
(pathom-parser {} [:instruments]) works as expected: {:instruments :com.wsscode.pathom.core/not-found}. (`reader3`, but the same occurs on reader2 and parallel-reader. Pathom version 2.3.0-alpha17)
#2020-11-1614:09nivekuilThink you need to quote the list#2020-11-1614:11henrikYup, thank you!#2020-11-1614:11henrikDoh, it’s a faux function expression…#2020-11-1817:53souenzzo@wilkerlucio I did a mutation that can handle "any" keys in it's params.
(pc/defmutation my-custom-mutation [_ params]
  {::pc/sym 'app/my-custom-mutation}
  (custom-stuff prams))
This it a bad practice//officially supported??
#2020-11-1820:00wilkerlucionothing wrong with that, pathom considerer maps open as default#2020-11-1820:01wilkerluciocurrently ::pc/params is purely for documentation sake (users can read from code, also shows up in index explorer)#2020-11-1820:01wilkerlucioin the future I wanna use params to auto-complete mutation options in the query editor#2020-11-1820:01wilkerluciobut regarding about pathom runtime, they don't affect anything#2020-11-1822:31wilkerlucioPathom 3 Doc Update: New section on Resolver Parameters - https://pathom3.wsscode.com/docs/resolvers/#parameters#2020-11-1915:06andrewzhurovhey guys is it possible to derive a RESTful API out of Pathom API? e.i. generate GET endpoints out of resolvers, POST endpoints out of mutations#2020-11-1918:16wilkerluciothere is no library, but its a trivial thing to achive, once you have the indexes, you can rub a query for a given endpoint, and let pathom to the resolution#2020-11-1918:16wilkerluciouse some http arg as param + some query#2020-11-1919:35souenzzoI do (almost) that. It's internal on the app that I work on. But works great. We do not "generate REST from reslvers" We write things like
{:endpoint "/user/:id"
 :query '[{[:app.user/id ?id] [:.appuser/name :app.user/id :app.user/description]}]}
#2020-11-2414:15frankitoxSo, I'm wondering how people handles HTTP status codes when using pathom. I'm using pedestal so I found some https://github.com/jlesquembre/pathom-pedestal/blob/18c27aa84526a8610e6f91247aae674723f599e5/src/pathom/pedestal.clj#L45-L49 that just always returns 200s. Is this a common practice in Pathom/GraphQL? For example, under certain auth conditions I'd like to return a 401 with a bit of info about the error.#2020-11-2418:11souenzzo@franquito in a #fulcro app, your communication with the server is via EQL, "not via HTTP" HTTP will return 200 always that "the EQL request has been processed" if your EQL Query has a not allowed access, you will return a 200 with a response that contains that "not allowed" message. But it's nothing about #pathom , it's more about "how we develop SPA in 2020" I use #pathom to deliver a REST API, returning different codes for each request#2020-11-2420:33frankitoxI see. I'm using re-frame only with the POST /api endpoint now (that's the basic implementation I've seen in the wild). Now I'm looking to add JWT to the mix, which means setting headers. In this REST API you reify the status codes in Pathom? For example, a resolver or a mutation may return a :response/status and you use that with the HTTP server?#2020-11-2420:43souenzzo@franquito a bit of my opinion When you send a query with a mutation with a mutation, if the mutation run¹, it will return 200. It say nothing about success/fail of the mutaion. it says about the success of execution of the query IMHO, the mutation should return something like {my/mutation {:fail {:msg "can't because x" :some-explain {..data..}}}} ¹ - mutation run: (parser env tx) => map? not trow#2020-11-2420:52souenzzoBut it's a open system. #fulcro is just one impl. My current pathom usage is a SSR app where I usa "raw" HTML form's to create POST's and programmatically turn these POST's into an EQL tx, return 303 -> Location and get a new page (rended by hiccup) I all "EQL tx" a "query that contains a mutation"#2020-11-2421:02frankitoxCrap, I'm still very confused :thinking_face: but I get your idea of 200s expressing a successful execution. The biggest question is how much of your system you end up representing with the keywords. I think I need more hammock time here, thank you for the helpful thoughts Enzzo!#2020-11-2421:14souenzzoThinking about "network stack": Your HTTP server do not send a TCP ACK when your application fails, it send's a TCP OK with a HTTP FAIL Your EQL server do not send 400 when the mutation fails, it send a HTTP OK with a ::p/errors#2020-11-2420:19wilkerlucioreleased [com.wsscode/pathom "2.3.0-alpha19"], this gives a fixes a problem with the jar, that previously included the compiled js data, the jar is reduced from 9.1MB to 103kb#2020-11-2517:18wilkerlucioNew Pathom3 documentation page out! This time is for placeholders: https://pathom3.wsscode.com/docs/placeholders#2020-11-2603:41wilkerlucioPathom 3 Mutations are out! https://pathom3.wsscode.com/docs/mutations#2020-11-2604:05cjmurphyFrom my experience inputs are useful with mutations. The need for them comes up quite often in this forum. Will the workaround still be the same, to find the parser in the env and call it again?#2020-11-2604:54wilkerlucioyes, but could be also a transform or a plugin, depending on which level you intend to configure it#2020-11-2605:37cjmurphyI like a really simple model, not having to configure things. I kind of hoped that would be the aim with Pathom 3.#2020-11-2617:09cjmurphyWhat's the maven artifact that has com.wsscode.pathom.viz.ws-connector.core in it? Just trying to get Pathom Viz going.#2020-11-2617:15wilkerlucio@cjmurphy https://github.com/wilkerlucio/pathom-viz-connector#2020-11-2617:16wilkerlucioPathom Viz usage docs: https://roamresearch.com/#/app/wsscode/page/RG9C93Sip#2020-11-2722:36wilkerlucioNew Pathom update post: https://blog.wsscode.com/pathom-updates-04/#2020-11-2722:38wilkerlucioI also opened a new Patreon page, if you or your company would like to support my open source work check it at: https://www.patreon.com/wsscode#2020-11-2804:34Vincent Cantin@wilkerlucio The pathom documentation site is missing an obvious link to the blog.#2020-11-3013:15wilkerlucioāœ…#2020-11-2907:45Thomas MoermanPlaceholders with params! :the_horns::skin-tone-3: parrot šŸŽ‰ šŸ”„ šŸ”„ šŸ”„#2020-11-2914:31wilkerluciothanks to @U2J4FRT2T for the ideia :)#2020-11-2918:25Vincent CantinIs there anybody working on a reactive implementation of pathom?#2020-11-3013:18wilkerlucioI don't think at this time. I was thinking about it, users could do a "query subscription" (kind like Fulcro does), and mutations could have a "affected attributes" declaration, from that, we could use Pathom index to figure out also all the computations affected by those attributes, and in case there is a subscription to any of that, refresh it#2020-11-3017:15rickmoynihanQuestion… is it possible to dynamically generate input and output properties / resolvers with pathom?#2020-11-3017:17rickmoynihanEssentially I need to discover the attributes for my properties at run time. I think I would need to query my system to discover them dynamically before I resolve the queries.
#2020-11-3017:18rickmoynihanIs there a recommended way to do such a thing with pathom? From what I’ve seen it looks like properties must be coded up front, rather than discovered from your data#2020-11-3017:32rickmoynihanok just seen there is a function resolver#2020-11-3017:33souenzzoI already generated resolvers from a wired//custom rest API/specification. Worked really well#2020-11-3017:35rickmoynihanyeah I think this would be sufficient… I haven’t played with pathom in any depth yet; just trying to assess at a high level whether I could make it work. My underlying data model is already open-world/RDF so in many ways it’s a natural fit; though I’d need to discover applicable attributes through meta-level queries first.#2020-11-3018:58myguidingstarcomputing the graph indexes is rather expensive, so I guess it'll help if they can be generated in meta-level phase to minimize overhead in subsequent requests#2020-11-3018:59wilkerlucioI dont think they are expensive to generate, but surely you dont wanna do it once before each query, but generating once at app start should be fine#2020-11-3019:09myguidingstarfor RDF kind of problem I guess the open-world is just too big (infinite?) to be indexed at start time but instead must be "lazily" expanded as the client asks for more. I can imagine keeping track of several (dynamically generated) child resolvers for each session (or even "garbage collecting" them somehow?)... Of course I don't know that much about rick's use case šŸ™‚#2020-11-3019:11myguidingstaralso, maybe it'll be an interesting case for pathom's query UI where the whole graph is not known beforehand?#2020-11-3021:05wilkerlucioyou can always cache the index if gets too heavy#2020-11-3021:05wilkerlucioand distributed environments (local or remote) could get somethink like expanding graph#2020-11-3021:05wilkerluciobut figuring index at planning stage is not something I see viable#2020-11-3021:11wilkerluciofor RDF, as same as GraphQL, and Datomic, there will be a better support for dynamic resolvers, this is where you can improve things, but you still have to know all the attributes ahead of time, otherwise pathom can't tell when to start processing or not#2020-11-3021:59myguidingstarhmm, I guess "have to know all the attributes ahead of time" can be a frown for rick, but let's wait for his confirmation. IIrc, usually you connect to one RDF endpoint and may get references to other RDF endpoints, and the number of endpoints is kinda infinite, that's why you can't know beforehand. Anyway, this also need confirmation#2020-12-0110:14rickmoynihan> hmm, I guess ā€œhave to know all the attributes ahead of timeā€ can be a frown for rick, Yeah it looks like this might be a problem for us šŸ˜ž > but generating once at app start should be fine This won’t work for us as they’d have to be rebuilt on every data update, and that really isn’t practical for a number of reasons. It would be far easier to discover them at query time Essentially our problem is that our platform hosts data that can essentially be anything, with arbitrary predicates that our users can coin and supply themselves. Obviously a generic platform can’t do anything bespoke without knowledge of the data, so we target the meta-level (vocabularies) which provide schemas around the shape of the data. However the vocabularies are a level removed from the actual properties themselves… For example one of the main vocabularies we use is the W3C RDF Data cube ( https://www.w3.org/TR/vocab-data-cube/ ) which essentially models multidimensional statistical data into dimensions, attributes, measures and observations. An observation is essentially just a cell in an N-dimensional spreadsheet triangulated by it’s dimensions… they usually include area and time but also arbitrary other dimensions specific to the cube, for example homelessness data might include dimensions on gender / age, but if someone loaded a trade dataset it would have imports, exports and dimension properties like ā€œchained volume measureā€ etc. We can’t feasibly know all of these when we write the software; but we can rely on them being formally described in that vocabulary; so we can discover what they are for any given cube at query time; either through the datasets Data Structure Definition (DSD) or via the fact that all dimension predicates are of type DimensionProperty. The DSD bit of the cube vocabulary is essentially a meta-schema for describing cube-schemas; but they are described in amongst the triples/data itself. So in order to use pathom I was hoping to be able to dynamically generate a subset of the resolvers at query time.#2020-12-0110:20rickmoynihanSo essentially I’d need to provide functions for things like ::pco/input and ::pco/output instead of hard coded vectors#2020-12-0110:20rickmoynihan
(pco/defresolver fetch-all-observations [{:keys [cube/id]}]
  {::pco/input  [:cube/id]
   ::pco/output #(lookup-cube-dimensions id)}
  (fetch-observations id))
#2020-11-3021:26Bjƶrn EbbinghausAm I right that the viz full graph doesn’t show edges on a to-many relationship?#2020-11-3021:41wilkerlucionot the nested parts, it connects with the attribute that has the list, but not the items (the indirect things)#2020-11-3021:48JAtkinsMaybe this is crazy/impossible, but can dynamic dependencies work? use case could be something like total-book-cost
pallet-cost, pallet-count, book-cost, book-count

total-book-cost:
  if present (pallet-cost, pallet-count) (* pallet-cost pallet-count)
  if present (book-cost, book-count) (* book-cost, book-count)
  else err

(total-book-cost {:pallet-cost 1 :pallet-count 2}) => 2

(total-book-cost {:book-cost 4 :book-count 4}) => 16
#2020-11-3021:49JAtkinsIt would (maybe?) be slow (having to trace if book-cost can be calculated from other params), but it could be useful if possible.#2020-11-3022:15souenzzo@jatkin It's possible and AFIK, it's pretty optional performance#2020-11-3022:16souenzzohttps://gist.github.com/souenzzo/5eb8387425212450519d0b754cc34445#2020-11-3022:17JAtkinsOh, duh. I'm an idiot šŸ™‚. I thought there was much more ceremony here.#2020-11-3022:18JAtkinsThanks very much for taking the time to write this out!#2020-11-3022:33souenzzo+pathom3 example on the same gist Pathom3 is way more simpler šŸ™‚#2020-11-3022:35wilkerluciojust a warning with this approach, in case the entry has access to all 4 attributes, the result becomes uncertain#2020-11-3022:38wilkerlucioone interesting thing that this case brings to mind is that you can also make a function to return resolvers, so another way to implement this:#2020-11-3022:38wilkerlucio
(defn attr-total-cost-resolver [attr]
  (let [cost-kw       (keyword (str attr "-cost"))
        count-kw      (keyword (str attr "-cost"))
        total-cost-kw (keyword (str "total-" attr "-cost"))
        sym           (symbol (str "total-book-cost-from-" attr))]
    [(pc/resolver sym
       {::pc/input  #{cost-kw
                      count-kw}
        ::pc/output [total-cost-kw]}
       (fn [_ input]
         {total-cost-kw (* (cost-kw input) (count-kw input))}))
     (pc/alias-resolver total-cost-kw :total-book-cost)]))

(let [;; Relevant part: the resolvers
      registers [(attr-total-cost-resolver "book")
                 (attr-total-cost-resolver "pallet")]
      ;; pathom2 parser. Pathom3 is simpler
      parser    (p/parser {::p/plugins [(pc/connect-plugin)]})
      env       {::p/reader               [p/map-reader
                                           pc/reader2
                                           pc/open-ident-reader
                                           env-placeholder-reader-v2] ;; I backported pathom3 placeholders to pathom2
                 ::pc/indexes             (pc/register {} registers)
                 ::p/placeholder-prefixes #{">"}}]
  (parser env `[{(:>/pallet {:pallet-cost 1 :pallet-count 2})
                 [:total-book-cost]}]))
#2020-11-3022:39wilkerluciobut still the same in consideration the same warning I said before#2020-11-3022:41wilkerlucioI think a proper solution to it requires optional inputs, that's something planned for pathom 3, so you can ask for all attributes (as optionals) e than make your logic inside the resolver, so you have more control#2020-11-3022:44wilkerlucioone way to archive this on pathom 2, is having a resolver that doesnt require any input, then from inside of it you call the parser again, and ask for the attributes (all of then), and work with this result, like:
(pc/defresolver total-book-cost [{:keys [parser] :as env} _]
  {::pc/output [:total-book-cost]}
  (let [result (parser env [:book-cost
                            :book-count
                            :pallet-cost
                            :pallet-count])]
    ...))
#2020-12-0110:29rickmoynihanAre there any examples of doing conjunctive and disjunctive queries in pathom? i.e. and and or. Presumably you need to implement this kind of thing as parameters? Can you nest and compose these parameters? e.g. maybe something like [(::and :prop/a :prop/b (::or :prop/c :prop/d)]#2020-12-0122:22dehlihello! is it possible to reset cache for a specific pathom query?#2020-12-0122:44Thomas MoermanYep, you can manipulate request-cache in env#2020-12-0122:51dehliawesome! thanks so much!#2020-12-0300:02genekimHello, @wilkerlucio — after listening to all your interviews with @jacek.schae, I was blown away, and watched all the videos you linked to. Congrats on continuing to build out such a remarkable and novel way managing data! After seeing the magic that’s possible, I suddenly find it quite distasteful to have to write API and database queries over and over again, and often, once on the client side, and another time on the server side! I may have been mildly annoyed about doing that before, but after seeing the SpaceX and YouTube demo that @pithyless did, I now know that it doesn’t have to be that way — and suddenly, writing the same queries in two portions of code seems intolerable! šŸ˜† šŸ˜† šŸ˜† I’ve spent about 6 hours trying to write a resolver against the Vimeo — it’s been kind of tough sledding, because it’s taken me awhile to comprehend sufficiently how all the pieces get created. I found myself watching the ā€œOm Nextā€ video from @wilkerlucio several times, pausing many times to watch how he built the Spotify example. I just published my work in progress here — my goal is to document my experiences trying to write a Vimeo adapter, point out where I had sticking points, and propose some doc changes. I’ll write that up when I’m a little further down, but wanted to share progress so far, just in case it’s useful (or amusing. šŸ™‚ I’ve written the easiest resolver (ā€œvideo-by-idā€), and I’m on the third attempt to get ā€œalbums-by-user-idā€ written — I got hung up on how to deal with the collection inside of the return value… https://github.com/realgenekim/pathom-demo/tree/gene-vimeo-hacking-away Happy to hear any critiques or recommendations on the work I’ve done so far — here’s to hoping I can get ā€œalbums-by-user-idā€ working in this work session tonight! Thank you, @wilkerlucio and @pithyless!#2020-12-0300:02genekim(Sorry, meant to post more, but need to run — but wanted to post something now, as opposed to more later. šŸ™‚#2020-12-0304:42genekimOkay, for the life of me, I can’t seem to get the Vimeo equivalent to the spotify.top-tracks to work, as was shown in this repo: https://github.com/wilkerlucio/spotify-pathom/blob/master/src/spotify_pathom/parser.clj#L64 I manually assoc in a vimeo.album-list.data -> vimeo.album/user-id, creating an alias with that and vimeo.user/id. And I just can’t figure out how to write the equivalent EQL of the spotify-top-tracks. How do I get all the album URLs for a given user? Blindly copying the spotify query, I feel like it should be something like in the screenshot below from the video. Any hints much appreciated!!! šŸ™šŸ™
#2020-12-0304:42genekimHere’s the relevant part of the 2017 ā€œbeyond Om Next video: https://youtu.be/60i9uStI9As?t=1216#2020-12-0311:57wilkerluciohello @genekim! happy to see your excitement šŸ™‚#2020-12-0312:05wilkerlucio@genekim not sure if I understand your question, can you send the example query you are trying to run? what you expect, and what is happening?#2020-12-0313:00souenzzo@wilkerlucio there is some function to remove the "cache" from env? I can do a "select-keys" and get just the index keys, but it will remove my custom env vars. Something like:
(pco/defresolver foo [env _]
  (let [... (p.eql/process (pci/without-cache env)
                           [::my-key])]
    ...))
#2020-12-0313:02wilkerlucio
(dissoc env
  :com.wsscode.pathom3.connect.runner/resolver-cache*
  :com.wsscode.pathom3.connect.planner/plan-cache*)
#2020-12-0313:03wilkerluciothis will create new sub caches on run, if you really want to disable cache entirely, then:
(assoc env
  :com.wsscode.pathom3.connect.runner/resolver-cache* nil
  :com.wsscode.pathom3.connect.planner/plan-cache* nil)
#2020-12-0313:03wilkerlucioI suggets keeping plan-cache*, but I mentioned here just for completeness#2020-12-0316:17genekim@wilkerlucio Thx for the focusing question. My query that isn’t working looks like
[{[:vimeo.user/id 118038002]
  [* :vimeo.album-list/data
   [[:vimeo.album/uri]]]}]
` I want the EQL query to return something with the shape:
{[:vimeo.user/id 118038002
  [:vimeo.album/uri "uri1"]
  [:vimeo.album/uri "uri1"]])
 
…but instead, I get the following… I can’t figure out how to query against anything inside the collection at vimeo.album-list/data. Thank you!!!
{[:vimeo.user/id 118038002]
 {:vimeo.album-list.paging/previous nil,
  :vimeo.album-list.paging/next nil,
  :vimeo.album-list.paging/last "/118038002/albums/?page=1",
  :vimeo.album-list/per-page 25,
  :vimeo.album-list.paging/first "/118038002/albums/?page=1",
  :vimeo.album-list/id 118038002,
  :vimeo.album-list/page 1,
  :vimeo.album-list/total 8,
  :vimeo.album-list/data
  [{:vimeo.album/uri "/users/118038002/albums/7670034",
    :vimeo.album/autoplay false,
    :vimeo.album.user/link "",
    :vimeo.album.metadata.interactions.add-logos/uri
    "/users/118038002/albums/7670034/logos",
    :vimeo.album.user.preferences.videos.privacy/view "disable",
    :vimeo.album.user.metadata.interactions.block/uri
    "/me/block/118038002",
    :vimeo.album.user.metadata.connections.channels/total 0,
    :vimeo.album.user.metadata.interactions.follow/added false,
    :vimeo.album.user.metadata.connections.teams/options ["GET"],
    :vimeo.album.user.metadata.connections.folders-root/uri
    "/users/118038002/folders/root",
    :vimeo.album.user.pictures/resource-key
    "9b6cd50b7543a03d77365da646638c8e06f4c7ea",
    :vimeo.album.user.metadata.connections.followers/options ["GET"],
    :vimeo.album.metadata.interactions.add-live-events/options
    ["POST"],
    :vimeo.album.user.metadata.connections.folders/uri
    "/users/118038002/folders",
    :vimeo.album.metadata.connections.videos/uri
    "/albums/7670034/videos",
    :vimeo.album/brand-color nil,
    :vimeo.album.user.metadata.connections.videos/options ["GET"],
    :vimeo.album.privacy/view "anybody",
    :vimeo.album.user.metadata.connections.appearances/total 0,
    :vimeo.album/seo-keywords [],
    :vimeo.album.user.metadata.connections.portfolios/uri
    "/users/118038002/portfolios",
    :vimeo.album/name "Shaaron",
    :vimeo.album.pictures/sizes
    [{:width 100,
      :height 75,
...
#2020-12-0316:20genekim…I can’t quite figure out what the EQL is for doing joins in collections… and I keep wondering whether I wrote the resolver incorrectly. Thx!#2020-12-0316:23pithylessJoins use the same syntax for to-one and to-many relationships; the way you tell them apart is by returning {:join-key {:a :b}} or {:join-key [{:a :b}]#2020-12-0316:24pithyless^ in both cases, the EQL would be the same: [{:join-key [:a]}]#2020-12-0316:21pithyless@genekim I think you're missing a join:
{[:vimeo.user/id 118038002]
 [{:vimeo.album-list/data [:vimeo.album/uri]}]}
#2020-12-0316:24genekimHoly cow — that’s cool! I’m getting this error in the parser — does this mean I’m missing a resolver? (PS: thanks for your talk on SpaceX/YouTube. It was what made me finally feel like I should jump in and try to make this work!!!)#2020-12-0316:27pithylessIt looks like the resolver indices are not set up correctly and can't resolve this relation#2020-12-0316:30pithyless@genekim does that :vimeo.album-list/data autocomplete?#2020-12-0316:30genekimGot it…. and this query suggests that, as well… Any chance you can see what I did wrong in either my attrs or resolvers? Resolver.cljs: https://github.com/realgenekim/pathom-demo/blob/gene-vimeo-hacking-away/src/demo/connect/vimeo/resolvers.cljs Attrs.cljs: https://github.com/realgenekim/pathom-demo/blob/gene-vimeo-hacking-away/src/demo/connect/vimeo/attrs.cljs#L283#2020-12-0316:31genekim@pithyless Something is definitely off — what does it mean when something displayed in red vs. brown in the parser window?#2020-12-0316:34pithylessThere's a lot going on here, try to simplify it for now:
(def albums-by-user-id
  (pc/defresolver albums-by-user-id [env input]
    {::pc/input  #{:vimeo.user/id}
     ::pc/output [{:vimeo.album-list/data [:vimeo.album/user-id]}]
     {:vimeo.album-list/data [{:vimeo.album/user-id "fake-user-id"}]}}))
#2020-12-0316:35pithylessDoes that work with this query?
{[:vimeo.user/id 118038002]
 [{:vimeo.album-list/data [:vimeo.album/uri]}]}
#2020-12-0316:37genekimOMG! It works! šŸŽ‰šŸŽ‰šŸ™šŸ™ THANK YOU! Holy cow! That means code is okay, right? This is fantastic!#2020-12-0316:38pithylesscool šŸ˜Ž#2020-12-0316:41genekimGotta step away from keyboard for an hour, but that query worked without the code change that you proposed — whew! …and can you explain the EQL syntax you used? In desperation, I thought I had tried every permutation of ā€œ[ā€ and ā€œ{ā€, and still somehow managed to miss this. Is there a section in the docs that explains this type of join against a collection?#2020-12-0316:43pithylessWhen debugging resolvers, two common things to look for: 1. is the autocomplete working and are the indices aware of the relation I'm trying to join 2. are the resolvers returning the right shape? (e.g. accidentally returning [{:name "a"}] instead of {:join-key [{:name "a"]} if the resolver output is supposed to be [{:join-key [:name]}]#2020-12-0316:44pithylessHave you seen the EQL docs? https://edn-query-language.org/eql/1.0.0/specification.html#_joins#2020-12-0316:53pithyless@genekim here's a step-by-step process how to think about the query you just wrote:
Start with a vector of attributes:

[SOME_ATTRS]

One of the attrs should be a join, so we need a map:

[{SOME_JOIN_KEY [SOME_JOIN_ATTRS]} ...POSSIBLY_MORE_ATTRS...]

In this case, we only care about the one attribute:

[{SOME_JOIN_KEY [SOME_JOIN_ATTRS]}]

What's the join key? An ident in this case:

[{[:vimeo.user/id 118038002] [SOME_JOIN_ATTRS]}]

What are the attributes we're interested in? Well, one of them is a join:

[{[:vimeo.user/id 118038002] [{NEW_JOIN_KEY [NEW_JOIN_ATTRS]} MORE_ATTRS]}]

Actually, we only care about that one join attribute:

[{[:vimeo.user/id 118038002] [{NEW_JOIN_KEY [NEW_JOIN_ATTRS]}]}]

What's the join key?

[{[:vimeo.user/id 118038002] [{:vimeo.album-list/data [NEW_JOIN_ATTRS]}]}]

What are we trying to pull from that relationship?

[{[:vimeo.user/id 118038002] [{:vimeo.album-list/data [:vimeo.album/uri]}]}]
Just rinse and repeat to build bigger queries šŸ™‚
#2020-12-0316:55pithylessAnd in case you missed the other thread response: Joins use the same syntax for to-one and to-many relationships; the way you tell them apart is by returningĀ `{:join-key {:a :b}}`Ā orĀ `{:join-key [{:a :b}]` In both cases, the EQL (and the resolver output) would be the same:Ā `[{:join-key [:a]}]`#2020-12-0317:58genekimWow! Thank you @pithyless! That is so helpful!!! I’ll be studying this for days to come, I’m sure. Much appreciated, and I’ll endeavor to write up my experiences when I get a little further, and maybe some suggestions on docs to make these more evident.#2020-12-0322:37wilkerlucio@genekim I would very much appreciate more ideas to improve the experience on early users šŸ™‚ both for the current pathom docs, and also for pathom 3, had you checked the docs over pathom 3? altough the usage has some differences, the fundamentals are the same, and the docs on pathom 3 reflect the most mature way I can explain those concepts#2020-12-0323:51genekimYep, I’ve read the current and Pathom 3 docs. FWIW, I think there’s one piece of ā€œgetting startedā€ docs missing, which would include the EQL docs that @pithyless just told me about today… …as well as more about the procedures of how to create the resolvers, the attributes, etc. I plan on drafting a HOWTO for my own purposes in that repo, so I can finish building out the Vimeo API. I’ll hopefully have something to share with you in about a week, which Iā€m hoping would be helpful to anyone who wants to get started. It’s super awesome to see my query work finally, and eager to now start pulling individual albums and videos next!#2020-12-0323:53genekim…I plan on documenting what it is I’m studying and copying when I watch certain portions of your ā€œOm Nextā€ video over and over again.#2020-12-0323:54genekimFor your amusement: šŸ˜‚ https://twitter.com/towernter/status/1332692092863340544#2020-12-0319:24genekimThanks again for your help, @pithyless! Does anyone have any advice on how to handle paginated response? As shown below, vimeo.album-list includes some information of how many pages are in the total response, URL of the next page, etc… How do people handle this? Do you pass in a page-number to vimeo.album-list? Do you load all the pages by default? (<-- I imagine this would be problematic for many scenarios. I’m imagining an API into Google Photos, which would load 100K entries upon every query. šŸ™‚ Are there any examples you’d point to of how others have handled pagination? Thanks again!#2020-12-0319:25genekim
{[:vimeo.user/id 118038002]
 {:vimeo.album-list.paging/previous nil,
  :vimeo.album-list.paging/next nil,
  :vimeo.album-list.paging/last "/118038002/albums/?page=1",
  :vimeo.album-list/per-page 25,
  :vimeo.album-list.paging/first "/118038002/albums/?page=1",
  :vimeo.album-list/id 118038002,
  :vimeo.album-list/page 1,
  :vimeo.album-list/total 8,
  :vimeo.album-list/data [{} {} ...]}
#2020-12-0319:58genekim…ah, it looks like the preferred method is passing in a cursor-like thing as a parameter to vimeo.album-list… I’m assuming this is still the case? https://clojurians-log.clojureverse.org/pathom/2018-12-10#2020-12-0323:54genekimFor your amusement: šŸ˜‚ https://twitter.com/towernter/status/1332692092863340544#2020-12-0405:41genekimOkay, I think I’m getting closer to having the big pieces stood up — I have a vimeo.video and vimeo-playlist , and some queries that run from Workspaces from inside a card. Now I’m trying to run those queries from inside my CLJS REPL, but all I get are errors…. All my code is here: https://github.com/realgenekim/pathom-demo/blob/gene-vimeo-hacking-away/src/demo/cards/workspaces_main.cljs#L59 Currently, the original demo code had a (defn parser [] (p/parallel-parser) , so it wouldn’t accept any arguments from the REPL. Copying the code, I just turned it into a def intparser [] (p/parallel-parser) . But the results from the REPL are shown below. I get the channel returned, but I can’t get the results out — `<!` results in a CLJS error.
(def x (intparser {} [{[:vimeo.user/id 118038002]
                         [{:vimeo.album-list/data [:vimeo.album/uri]}]}]))

; => #object[cljs.core.async.impl.channels.ManyToManyChannel]
;
; that worked, right?  so, how do I get the result of the query out?

(go (async/<! (println x)))
(go-catch (<? x))

; TypeError: Cannot read property 'call' of undefined
;    at eval (eval at <anonymous> (), <anonymous>:1:31)

(go (async/<! (intparser {} [{[:vimeo.user/id 118038002]
                              [{:vimeo.album-list/data [:vimeo.album/uri]}]}]))))

; TypeError: Cannot read property 'call' of undefined
;    at eval (eval at <anonymous> (), <anonymous>:1:31)
How do I get the query results? Thx! (Am I setting up the parser incorrectly? Am I using async wrong?)
#2020-12-0411:32wilkerlucionot sure whats wrong, but I noticed a few things: - you should not need to set ::pc/resolver-dispatch and ::pc/mutate-dispatch - I encourage you to prefer the async parser instead of the parallel one when learning, the parallel is way more complicated and given the overhead it adds only a few cases get real improvements from it (and if you change, remember to also change the parallel-reader to async-reader2)#2020-12-0411:52wilkerlucioI tried it here, for some reason it fails with go, but works with go-catch:#2020-12-0411:52wilkerlucio
(go-catch
    (js/console.log
      (<?
        (intparser {} [{[:vimeo.user/id 118038002]
                        [{:vimeo.album-list/data [:vimeo.album/uri]}]}]))))
#2020-12-0504:04genekimThx @U066U8JQJ — I switched from parallel reader to async-reader2, and now I get a result, but it has a channel in it. > (defn q [] > (intparser {} [{[:vimeo.user/id 118038002] > [{:vimeo.album-list/data [:vimeo.album/uri]}]}])) > > (q) > => {[:vimeo.user/id 118038002] > {:vimeo.album-list/data #object[cljs.core.async.impl.channels.ManyToManyChannel]}}#2020-12-0504:06wilkerluciodid you changed the parser to use async-parser too? it should return a channel from the parser call#2020-12-0504:07genekimOh, gosh. I think I tried that once. Let me try that right now…. (What I had tried is here: https://github.com/realgenekim/pathom-demo/blob/36caaa257ed830d6c125c6acf603829cd18706b4/src/demo/cards/workspaces_main.cljs#L59)#2020-12-0504:09genekim(PS: is the reason I’m having so many problems is that I’m using an outdated basis to build upon? I chose the @U05476190 repo from years ago… Perhaps there was a more appropriate sample to use? Okay, trying async-parser now! Thx for this help!)#2020-12-0504:12genekimSorry, wrong github link: https://github.com/realgenekim/pathom-demo/blob/gene-vimeo-hacking-away/src/demo/cards/workspaces_main.cljs#2020-12-0504:13genekimNow I’m getting a reader error… Hmm…. Uncommenting out some of what I took out… > > {[:vimeo.user/id 118038002] {:vimeo.album-list/data :com.wsscode.pathom.core/reader-error}, :com.wsscode.pathom.core/errors {[[:vimeo.user/id 118038002] :vimeo.album-list/data] Cannot read property ā€˜cljs$core$IFn$_invoke$arity$1’ of null}}#2020-12-0504:53genekimOMG. It works!!! Thanks so much for the help, @U066U8JQJ!!!I > {[:vimeo.user/id 118038002] {:vimeo.album-list/data [{:vimeo.album/uri ā€œ/users/118038002/albums/7670034ā€} > {:vimeo.album/uri ā€œ/users/118038002/albums/7668043"} > {:vimeo.album/uri ā€œ/users/118038002/albums/7657922ā€} > {:vimeo.album/uri ā€œ/users/118038002/albums/7657919"} > {:vimeo.album/uri ā€œ/users/118038002/albums/7425464ā€} > {:vimeo.album/uri ā€œ/users/118038002/albums/7299521"} > {:vimeo.album/uri ā€œ/users/118038002/albums/7273701ā€} > {:vimeo.album/uri ā€œ/users/118038002/albums/7266618"}]}}#2020-12-0517:21Adrian SmithAny idea why I'm getting null for my arguments in this resolver? I'm working from fe66c21ad37cba48c6ceba3cdc240a0092f78fe8 (pathom 3)#2020-12-0517:30Adrian SmithThe resolver call is at the bottom of the screenshot#2020-12-0521:24souenzzo@UCJCPTW8J as a "batch' resolver, it will always receive/return a "coll-of <input keys>)#2020-12-0613:45Reshef MannHi. Any suggestions for using batch resolver for the N+1 problem when using a database. The way I see it, in order to "reuse" resolvers when querying a remote db (say, user_group->user), if I have a user_group resolver and a user resolver, I will get hit by the N+1 problem when returning only {:user/id 2222} in the user_group resolver. I guess I can create a query that performs the join inside the user_group resolver but that causes duplication and looses the pathom magic.#2020-12-0619:28souenzzo@reshef.mann pathom (alone) do not solve n+1 problem There is projects like #walkable that try to help about it. There is cool ideas here too (not directly related to pathom, but I believe that this ideas can help with a solution): https://blog.jooq.org/2019/11/13/stop-mapping-stuff-in-your-middleware-use-sqls-xml-or-json-operators-instead/ I think that is important to remember that in modern db's like #datomic , n+1 isn't a problem at all, once every access is in-memory.#2020-12-0711:52Reshef Mann@U2J4FRT2T Thanks for the helpful info! It was more out of curiosity. For the project I'm talking about I'm using Crux so the same traits of datomic apply as well.#2020-12-0714:06souenzzoIn my experience, using datomic, even on 3-level joins (list some users, for each user get all addresses, for each address get something else), i have no performance problem (once everything is on db).#2020-12-0619:32souenzzo@reshef.mann A practical approach, that can be used to solve not only n+1, but any performance problem in #pathom: - Develop your app, write small/simple/naive resolvers to connect/deliver your app - Deploy your product, let the client's do requests - Collect statistics about which queries and which resolvers are slow//called too much (pathom trace will help) - Create specialized batch resolvers for these situations - Pathom will smartily choose the faster resolver for the query.#2020-12-1019:50wilkerluciohello everyone, just to let you know I setup the Discussions for Pathom in Github, yet another place if you want to send more durable ideas and etc: https://github.com/wilkerlucio/pathom3/discussions#2020-12-1020:12wilkerlucionew Pathom 3 docs page on Planner! I'm very excite to share this with you, it still a first version of it and there are still a lot to cover, but the current page can demonstrate the basics of how planning works on Pathom 3, it includes an interactive tool so you can see the planning happening step by step, I hope this can help you understand how pathom does its core behavior, check at: https://pathom3.wsscode.com/docs/planner I hope you enjoy!#2020-12-1115:14bbssreally really cool! I tried to use the dotfile graphs in one of the namespaces to explore it a bit, but couldn't make much sense out of it. This web-app thing is awesome!#2020-12-1115:21bbssI can imagine that being super useful when you can add durations to the edges for debugging performance.#2020-12-1214:43wilkerlucioyes, and this data is available after running, but what we are seeing now is just the planning step šŸ™‚#2020-12-1023:42dehliIs it possible to have defmutation errors also update the ::p.core/errors key?#2020-12-1106:26wilkerlucioyou can do with a custom ::p/process-error:#2020-12-1106:27wilkerlucio
(def mutation-parser
  (p/parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/reader2
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}
                  ::p/process-error        (fn [env err]
                                             (if (= :call (-> env :ast :type))
                                               (p/add-error (-> env
                                                                (update ::p/path p.misc/vconj (-> env :ast :key))
                                                                (dissoc ::p/process-error)) err))
                                             (p/error-message err))}
     ::p/mutate  pc/mutate
     ::p/plugins [(pc/connect-plugin {::pc/register mutation-error})
                  p/error-handler-plugin
                  p/request-cache-plugin
                  p/trace-plugin]}))
#2020-12-1112:24dehliawesome! thanks so much šŸ™‚ is it cool for me to create a github issue with the question and then copy you solution there for more visibility?#2020-12-1114:41wilkerluciosure, I think this is a good opportunity to try: https://github.com/wilkerlucio/pathom/discussions#2020-12-1100:44dehliAnother question, is it common/encouraged to have system only resolvers? I have some system states that I’m thinking could be easier to debug/dev if they were represented with resolvers but then i’d need to prevent clients from requesting them directly. One idea i’m running through is having a resolver that returns the database model and then downstream resolvers could take the database model and convert to the application model#2020-12-1100:54souenzzoI have resolvers that returns db schema, jvm startup data and I'm working//planning resolvers that returns my #pedestal routes and many more (use pathom as a "startup/component" system) For my API, I just filter some keys#2020-12-1100:56dehlithanks! ya it seems like fully embracing pathom is very powerful#2020-12-1100:59souenzzoIt's hard to code without pathom after use it for a while#2020-12-1115:38Aleedwith pathom3, what’s the correct way to set data (like a db connection) on env context?#2020-12-1115:42Aleedah I see that smart maps and eql process both take initial data#2020-12-1115:42Aleedwas trying to use the built-in resolvers at first šŸ˜…#2020-12-1115:51Aleed@U066U8JQJ seems like initial data is not passed to data map argument that mutations take? For arity two, inspecting env context I do see the initial data, but not sure which way I’m supposed to be accessing it.#2020-12-1115:52wilkerluciowhen create Smart Maps or EQL trigger, the first argument is the env, you should set the db connection there, same for mutations#2020-12-1115:52wilkerluciolike: (psm/smart-map (assoc env ::my-db ...) {:my.customer/id "123"})#2020-12-1115:56wilkerluciomakes sense?#2020-12-1115:56Aleedso pathom-indexes/register creates an env (not just indexes), and we can assoc additional env values into it before calling smart-map or eql process#2020-12-1115:57Aleedi’m trying this
(pathom-eql/process
   (assoc indexes
          ::ctx/conn (dc/conn))
   eql))
but assoc’ed data does not seem to be included in map passed to mutations
#2020-12-1115:59wilkerluciothere are two maps, the env and params#2020-12-1115:59Aleedhm i guess i have to use arity two to access it#2020-12-1115:59wilkerlucioto use env you need to declare the mutation with both#2020-12-1115:59Aleed^ yeah, i thought since everything was namespaced it’d be in one map#2020-12-1116:00wilkerlucio(pco/defmutation mmmm [env params])#2020-12-1116:00wilkerlucionope, two different contexts of information#2020-12-1120:03wilkerlucio#2020-12-1120:04wilkerlucioparts of Pathom Viz integrated in Reveal#2020-12-1120:57jmayaalvThis is awesome!! :star-struck::star-struck::star-struck:#2020-12-1223:40markaddlemanI'm starting to look at Pathom3. I notice that resolver exceptions are swallowed. This is just an unfinished piece of p3 or is there some magic I should put into the environment so that exceptions are percolated?#2020-12-1223:51markaddlemani see that sm-get-with-stats provides a way of obtaining the exception#2020-12-1303:09markaddlemanalso, https://pathom3.wsscode.com/docs/smart-maps/#error-modes#2020-12-1316:27wilkerluciohello, its an unfinished piece, in Pathom I've seen different ways people need error information, the smart map for example, since it does a "per attribute" request, its possible to have the loud error mode, but for EQL the answer may vary. the first thing to consider is that "partial errors" are a big thing, since pathom has control over the process a lot can happen inside, and how to report the failure depends on your app needs. For instance, for front-end applications, its probably interesting to check at attribute level to see what worked and what failed. but for monitoring you may wanna know every error that happen. there is also the case of errors in OR branches, where an error may had happened, but another path worked, in this case, this should be reported as an error? sorry the big blog of text, I'm dumping the current thoughs on the error handling on Pathom, feedback and ideas are very welcome#2020-12-1316:28wilkerlucioso my guess its will end up with a few differnet API to handle the different cases, so its silent by default, but with accessors for different needs#2020-12-1319:51markaddlemanNo worries. I appreciate the insight. I certainly see the need for multiple approaches for errors.#2020-12-1305:02markaddlemanIs there an equivalent of ::psm/error-mode-loud for eql?#2020-12-1316:55bbssin pathom3 are params different from eql? Here I'd expect :param to be available in the resolver.
(edn-query-language.core/query->ast
 ['({[:thing/id #uuid "c5aa81d4-13cf-491d-ac83-3646c9e0bc0b"]
     [:thing/attr]} {:param "param for thing"})])

;; => {:type :root, :children [{:type :join, :dispatch-key :thing/id, :key [:thing/id #uuid "c5aa81d4-13cf-491d-ac83-3646c9e0bc0b"], :query [:thing/attr], :children [{:type :prop, :dispatch-key :thing/attr, :key :thing/attr}], :params {:param "param for thing"}, :meta {:line 251, :column 4}}]}

(pco/defresolver resolver-using-param [env _]
  {::pco/input [:thing/id]
   ::pco/output [:thing/attr]
   ::pco/params [:param]}
  {:thing/attr (str "attr with " (:param (pco/params env)))})

(resolver-using-param
 {}
 ['({[:thing/id #uuid "c5aa81d4-13cf-491d-ac83-3646c9e0bc0b"]
     [:thing/attr]} {:param "param for thing"})])
#2020-12-1317:54wilkerluciothe parameter seems out of place in the query#2020-12-1317:54wilkerluciotry like this:
(edn-query-language.core/query->ast
 ['{[:thing/id #uuid "c5aa81d4-13cf-491d-ac83-3646c9e0bc0b"]
     [(:thing/attr {:param "param for thing"})]}])
#2020-12-1403:48bbssOk thanks. That is how a load with extra :params option from fulcro gave it, is it a bug there?#2020-12-1404:03bbssAnyway got it to work with that, used :update-query to add the param. Thanks again!#2020-12-1409:32wilkerlucionot a bug per-see, but an error in expectation, but its such a common one that I'm considering a way to give an easy fix, I see a lot of people doing plugins that just move the params forward everywhere, I think that's overkill, but I also think it could be something like "forward ident params", to move just params from idents to the first level of things#2020-12-1505:13bbssThat might be a nice extension, but if that somehow pollutes the params idea maybe a "global params" thing in the env would be better. That's what :params means in this case I think?#2020-12-1317:14yendaI tried updating from pathm 2.3.0 alpha 10 to 21 and I got a validation error in one of my resolver. it's responding to a union query and used to work fine so I don't understand the error. This is what it prints:
[{:follows-threshold [:notification/type :notification/timestamp :notification/text], :follow [:notification/type :notification/timestamp :user/id], :comment [:notification/type :notification/timestamp :comment/text :user/id :video/id], :comment-reply [:notification/type :notification/timestamp :comment/text :user/id :video/id], :like [:notification/type :notification/timestamp :user/id :video/id :video/deleted?], :video-uploaded [:notification/type :notification/timestamp :notification/text :video/id], :views-threshold [:notification/type :notification/timestamp :notification/text :video/id], :likes-threshold [:notification/type :notification/timestamp :notification/text :video/id]}] - failed: keyword? in: [:com.wsscode.pathom.connect/output 1] at: [:com.wsscode.pathom.connect/output :attribute-list :plain] spec: :edn-query-language.core/property
[{:follows-threshold [:notification/type :notification/timestamp :notification/text], :follow [:notification/type :notification/timestamp :user/id], :comment [:notification/type :notification/timestamp :comment/text :user/id :video/id], :comment-reply [:notification/type :notification/timestamp :comment/text :user/id :video/id], :like [:notification/type :notification/timestamp :user/id :video/id :video/deleted?], :video-uploaded [:notification/type :notification/timestamp :notification/text :video/id], :views-threshold [:notification/type :notification/timestamp :notification/text :video/id], :likes-threshold [:notification/type :notification/timestamp :notification/text :video/id]}] - failed: map? in: [:com.wsscode.pathom.connect/output 1] at: [:com.wsscode.pathom.connect/output :attribute-list :composed] spec: :com.wsscode.pathom.connect/out-attribute
[:pagination/edges [{:follows-threshold [:notification/type :notification/timestamp :notification/text], :follow [:notification/type :notification/timestamp :user/id], :comment [:notification/type :notification/timestamp :comment/text :user/id :video/id], :comment-reply [:notification/type :notification/timestamp :comment/text :user/id :video/id], :like [:notification/type :notification/timestamp :user/id :video/id :video/deleted?], :video-uploaded [:notification/type :notification/timestamp :notification/text :video/id], :views-threshold [:notification/type :notification/timestamp :notification/text :video/id], :likes-threshold [:notification/type :notification/timestamp :notification/text :video/id]}] :pagination/has-prev? :pagination/has-next? :pagination/first-cursor :pagination/last-cursor] - failed: map? in: [:com.wsscode.pathom.connect/output] at: [:com.wsscode.pathom.connect/output :union] spec: :com.wsscode.pathom.connect/output
#2020-12-1317:56wilkerlucioI can't understand from this, can you send the resolver output that's failing?#2020-12-1321:22yenda@U066U8JQJ
::pc/output [:pagination/edges [{:follows-threshold [:notification/type
                                                        :notification/timestamp
                                                        :notification/text]
                                    :follow [:notification/type
                                             :notification/timestamp
                                             :user/id]
                                    :comment [:notification/type
                                              :notification/timestamp
                                              :comment/text
                                              :user/id
                                              :video/id]
                                    :comment-reply [:notification/type
                                                    :notification/timestamp
                                                    :comment/text
                                                    :user/id
                                                    :video/id]
                                    :like [:notification/type
                                           :notification/timestamp
                                           :user/id
                                           :video/id
                                           :video/deleted?]
                                    :video-uploaded [:notification/type
                                                     :notification/timestamp
                                                     :notification/text
                                                     :video/id]
                                    :views-threshold [:notification/type
                                                      :notification/timestamp
                                                      :notification/text
                                                      :video/id]
                                    :likes-threshold [:notification/type
                                                      :notification/timestamp
                                                      :notification/text
                                                      :video/id]}]
                :pagination/has-prev?
                :pagination/has-next?
                :pagination/first-cursor
                :pagination/last-cursor]
#2020-12-1410:06wilkerlucio@U0DB715GU this union is malformed, for unions you should put the {} right after the join, without the [], as:#2020-12-1410:07wilkerlucio
::pc/output [:pagination/edges {:follows-threshold [:notification/type
                                                    :notification/timestamp
                                                    :notification/text]
                                :follow            [:notification/type
                                                    :notification/timestamp
                                                    :user/id]
                                :comment           [:notification/type
                                                    :notification/timestamp
                                                    :comment/text
                                                    :user/id
                                                    :video/id]
                                :comment-reply     [:notification/type
                                                    :notification/timestamp
                                                    :comment/text
                                                    :user/id
                                                    :video/id]
                                :like              [:notification/type
                                                    :notification/timestamp
                                                    :user/id
                                                    :video/id
                                                    :video/deleted?]
                                :video-uploaded    [:notification/type
                                                    :notification/timestamp
                                                    :notification/text
                                                    :video/id]
                                :views-threshold   [:notification/type
                                                    :notification/timestamp
                                                    :notification/text
                                                    :video/id]
                                :likes-threshold   [:notification/type
                                                    :notification/timestamp
                                                    :notification/text
                                                    :video/id]}
             :pagination/has-prev?
             :pagination/has-next?
             :pagination/first-cursor
             :pagination/last-cursor]
#2020-12-1519:06royalaidHey all, what is the best pratice around handling a very general operation like dissocing a key via pathom?#2020-12-1520:00souenzzoWhat you are trying to do? Can't you just dissoc from final result?#2020-12-1521:17royalaidA mutation in the db, we want to actually dissoc from the data at rest#2020-12-1521:19royalaidand we can’t depend the absence of a key corresponding to an implied dissoc in the case of an update because partial updates are possible#2020-12-1521:20royalaidso client sends up (entity/update {:e/uuid "XXX" :e/joined-obj :pathom.ops/dissoc})#2020-12-1521:20royalaidand it would remove the :e/joined-obj key from the object at rest#2020-12-1519:07royalaidCurrent ideas we have had were using something like an op value such as ::pathom.op/dissoc or overloading nil#2020-12-1615:03wilkerlucio#2020-12-1616:33tony.kayHey there, I’m seeing 5000 calls/minute for some props in pathom in prod. Any tips on what that might be? Used to be cache plugin missing kind of thing, but I think cache is built in now.#2020-12-1616:37wilkerlucionot sure, my first guess would be some collection processing?#2020-12-1616:38wilkerluciothe cache didn't changed in a long time#2020-12-1617:57tony.kaycool, I bet they didn’t code batch mode#2020-12-1617:57tony.kaynow that I think of it#2020-12-1719:22tony.kayI’ve got a pathom performance problem. I’m hitting a resolver like this:
(defresolver all-products-resolver
  [{:keys [conn ast] :as env} _]
  {::pc/output [{:product/all-products [:product/id
                                        :product/sku
                                        :product/description
                                        :product/precise-price
                                        {:taxable/taxes [:tax/rate :tax/category]}
                                        {:product/tax [:tax/rate :tax/category]}
                                        :product/inventory-tracked?]}]}
  (let [db         (d/db conn)
        company-id (env->effective-company-id env)]
    {:product/all-products (get-all-products db company-id)}))
and the query function on the last line returns exactly what the query asks for. The incoming query is exactly what this resolver outputs. When I measure this the get-all-products takes 680ms, and Pathom takes another 6s to process the result…and it invokes readers 10000 times (I have 10000 products) for every attribute…though those calls are instantly satisfied by map-reader they still consume about 3s of CPU. How do I optimize this to stop abusing Pathom (this is version 2.2.30. I tried 2.3.0 and it was much much much worse for some reason)
#2020-12-1719:22tony.kayI was hoping that satisfying the exact query would make pathom just return the result#2020-12-1719:24tony.kayPerhaps this is the kind of thing you’re restructuring in Pathom 3 šŸ˜„#2020-12-1719:39tvaughanA total shot in the dark, but would it help just to specify :product/all-products in the output, like ::pc/output [:product/all-products]? We have a query that has an output that is a map and we don't specify any of the sub-keys. We only query for the top-level key.#2020-12-1720:03wilkerlucio@tony.kay there is a escape feature for this situation, add the metadata ::p/final true to the list (when returning from the resolver), by doing so Pathom will skip processing it completly#2020-12-1721:40tony.kaythanks, I had a vague memory there was an escape hatch, but could not remember#2020-12-1722:38tony.kayno go. Does it matter what order the readers are installed?#2020-12-1722:39tony.kayI added debug breakpts in pathom, and only saw lots of calls to nested resolution..never saw that part of the program ever get the top-level list…#2020-12-1722:47tony.kayMap reader’s where I found that ::p/final, and when I watch what it does, I only ever see the individual nested entities. It never sees the list, so never skips it#2020-12-1723:46tony.kayYeah, I’m getting into here in connect:
(defn- process-simple-reader-response [{:keys [query] :as env} response]
  (let [key (-> env :ast :key)
        x   (get response key)]
    (cond
      (and query (sequential? x))
      (->> (mapv atom x) (p/join-seq env))

      (nil? x)
      (if (contains? response key)
        nil
        ::p/continue)

      :else
      (p/join (atom x) env))))
#2020-12-1723:47tony.kaykey is :products/all-products, and x is my list of 10,000 things, but it ignore metadata here and just processes them all anyway…unless, the query is nil#2020-12-1723:56wilkerluciothe thing that takes care of it is the map-reader#2020-12-1723:57wilkerluciohttps://github.com/wilkerlucio/pathom/blob/214834442615352df15c854e44d537bf2ced96a2/src/com/wsscode/pathom/core.cljc#L758#2020-12-1723:57tony.kayI’m tracing code with breakpoints#2020-12-1723:57tony.kaythat is where I land, and then it does all 10000#2020-12-1723:58wilkerlucioI can't find process-simple-reader-response in pathom, isn't this in your codebase?#2020-12-1723:58tony.kay#2020-12-1723:58tony.kayconnect.cljc#2020-12-1723:58tony.kayv 2.2.30#2020-12-1723:58wilkerlucioah, right#2020-12-1723:59wilkerluciodid you tried bumping to 2.3.0? I think that fn got removed#2020-12-1723:59tony.kayno, but 2.3.0 was waaaaay slower#2020-12-1723:59tony.kaycan’t risk that in prod#2020-12-1723:59tony.kayI guess I could try and diagnose that, but I’ve already spent 5 hours on what should have been easy optimizations šŸ˜„#2020-12-1800:00wilkerluciook, let me check on older version to understand#2020-12-1800:00tony.kayI’ll try 2.3 and see if there was something weird#2020-12-1800:03tony.kayok, weird#2020-12-1800:03tony.kaypathom 2.3 seems faster#2020-12-1800:03tony.kay@wilkerlucio#2020-12-1800:03tony.kaylet me eval this#2020-12-1800:04tony.kayYeah, I got one call this time#2020-12-1800:04wilkerluciocool#2020-12-1800:04tony.kayI wonder if it was slower earlire for some other reason#2020-12-1800:04tony.kayI’ll measure some more. Thanks for the help šŸ™‚#2020-12-1800:04wilkerlucioif that gets to a hard problem I can release a hotfix for 2.3.x with a fix for that fn you pointed out#2020-12-1800:04tony.kayI didn’t have the metadata on there earlier#2020-12-1800:05tony.kayso, I do think 2.3 is possibly slower without it. Let me check that#2020-12-1800:05wilkerlucioyeah, if you learn what it is, I would be glad to address, I don't remember any big change that could cause noticeable performance degradation#2020-12-1800:06tony.kayyeah…it’s waaaaaay worse#2020-12-1800:06wilkerluciobut maybe was this change, from moving to inline re-processing to the reader engine#2020-12-1800:06tony.kayI was getting 8s#2020-12-1800:06tony.kayit’s been running for 50s so far#2020-12-1800:06tony.kayand I was getting 2.5s with 2.2.30 with the metadata#2020-12-1800:07tony.kaystill running!#2020-12-1800:07tony.kay
q: :taxable/taxes                     536   504.60μs    30.54ms    54.41ms    57.27ms    60.14ms    65.60ms    30.09ms  ±50%    16.13s     71%
q: :product/tax                       535   188.04μs    11.61ms    21.51ms    22.96ms    25.35ms    29.50ms    12.01ms  ±51%     6.43s     28%
q: :entity/company                    536    34.74μs    52.49μs    72.69μs    96.27μs   128.20μs   359.06μs    56.05μs  ±23%    30.04ms     0%
q: :tax/category                      535    18.01μs    35.83μs    45.29μs    55.18μs    85.55μs   214.91μs    36.65μs  ±23%    19.61ms     0%
q: :product/precise-price             535    14.47μs    24.50μs    38.12μs    42.49μs    86.47μs   414.02μs    28.26μs  ±31%    15.12ms     0%
q: :tax/rate                          535    13.50μs    20.57μs    38.35μs    44.99μs    71.57μs   106.56μs    25.30μs  ±35%    13.53ms     0%
q: :product/inventory-tracked?        536    12.16μs    20.13μs    36.27μs    41.60μs    77.72μs   103.24μs    23.69μs  ±35%    12.70ms     0%
q: :product/description               535    12.71μs    19.32μs    35.91μs    38.43μs    74.16μs   102.37μs    23.24μs  ±35%    12.43ms     0%
q: :product/id                        536    11.15μs    18.36μs    35.35μs    41.61μs    76.47μs   127.77μs    22.69μs  ±38%    12.16ms     0%
q: :product/sku                       535    11.39μs    19.33μs    35.35μs    38.69μs    77.51μs   104.34μs    22.38μs  ±36%    11.98ms     0%
#2020-12-1800:07tony.kayI have minute-by-minute timing in a plugin#2020-12-1800:07tony.kaymeasuring every call to the reader#2020-12-1800:08tony.kayIt can normally do 10,000 of those in a few ms…but with 2.3 it is 10x that#2020-12-1800:08tony.kayor more#2020-12-1800:08tony.kayI’m seeing 11ms for 500 calls#2020-12-1800:09wilkerlucioI think the reason may be on the switch from inline processing to reader engine, but I wouldn't expect to be that much#2020-12-1800:09tony.kaystill running#2020-12-1800:09tony.kayI think it may be 100 to 1000x slower#2020-12-1800:10tony.kayyeah, I def cannot risk that one in prod šŸ˜•#2020-12-1800:10tony.kaythis would kill us#2020-12-1800:10wilkerlucioif you need I can do a hotfix for that 2.2 to add support for final there#2020-12-1800:10tony.kayyes please šŸ™‚#2020-12-1800:10tony.kayrepro case is easy#2020-12-1800:11tony.kay
(defresolver all-products-resolver
  [{:keys [conn] :as env} _]
  {::pc/output [{:product/all-products [:product/id
                                        :product/sku
                                        :product/description
                                        :product/precise-price
                                        :taxable/taxes
                                        :product/tax
                                        :product/inventory-tracked?]}]}
  (let [db         (d/db conn)
        company-id (env->effective-company-id env)]
    {:product/all-products (get-all-products db company-id)}))
#2020-12-1800:11tony.kayand make get-all-products return 1000 (complete) things#2020-12-1800:13wilkerlucioplease try 2.2.31-SNAPSHOT#2020-12-1800:13tony.kayk#2020-12-1800:13tony.kaythanks#2020-12-1800:14tony.kay(it’s still running btw)#2020-12-1800:14wilkerluciothis is running on CLJ, right?#2020-12-1800:14tony.kayyeah#2020-12-1800:14wilkerluciobecause I wanna do some measurements later, k#2020-12-1800:14tony.kay8700k CPU#2020-12-1800:15wilkerlucio8700k?#2020-12-1800:15tony.kayIntel CPU model#2020-12-1800:15tony.kaynot the latest, but no slouch#2020-12-1800:15wilkerlucioah, ok, just wanted to make sure I'm doing the tests in the same env (clj vs cljs)#2020-12-1800:15tony.kaysure#2020-12-1800:17wilkerlucioplease let me know if that works, if so I'll release 2.2.31#2020-12-1800:18tony.kayI had to restart, and this thing is rather large…takes a min#2020-12-1800:19wilkerluciono worries and no rush šŸ™‚#2020-12-1800:19tony.kay
cognitect.transit/write              transit.clj:  167
                 com.cognitect.transit.impl.WriterFactory$1.write       WriterFactory.java:   62
                      com.cognitect.transit.impl.JsonEmitter.emit         JsonEmitter.java:   41
            com.cognitect.transit.impl.AbstractEmitter.marshalTop     AbstractEmitter.java:  211
               com.cognitect.transit.impl.AbstractEmitter.marshal     AbstractEmitter.java:  184
               com.cognitect.transit.impl.AbstractEmitter.emitMap     AbstractEmitter.java:   85
                   com.cognitect.transit.impl.JsonEmitter.emitMap         JsonEmitter.java:  171
               com.cognitect.transit.impl.AbstractEmitter.marshal     AbstractEmitter.java:  194
       java.lang.Exception: Not supported: class clojure.lang.Atom
java.lang.RuntimeException: java.lang.Exception: Not supported: class clojure.lang.Atom
#2020-12-1800:19tony.kayooops#2020-12-1800:19tony.kayatom flowed through#2020-12-1800:20wilkerlucioah, I know what is#2020-12-1800:20wilkerluciosorry, minor mistake, releasing again#2020-12-1800:20tony.kayI should have cloned and run local/root so I could pull and jsut refresh instead of restarting šŸ˜•#2020-12-1800:20wilkerluciore-released#2020-12-1800:21wilkerluciohttps://github.com/wilkerlucio/pathom/tree/hotfix-final-2.3#2020-12-1800:21tony.kayk#2020-12-1800:21wilkerlucioits on this branch if you wanna pull local#2020-12-1800:22tony.kaydid u push?#2020-12-1800:23wilkerlucioyes#2020-12-1800:23tony.kayk…just a min#2020-12-1800:23wilkerlucioarg, it trigerred reformat, messy diff, going to fix, but the code should be correct#2020-12-1800:24wilkerlucioclean diff now: https://github.com/wilkerlucio/pathom/commit/2b1a4c368d9fdb170b31f28531608ad92d1eecce#2020-12-1800:26tony.kaythat got it#2020-12-1800:26tony.kayfast#2020-12-1800:27tony.kaynetwork request now 800ms#2020-12-1800:27tony.kayincluding data transfer#2020-12-1800:29wilkerluciocool#2020-12-1800:29wilkerlucioI'm doing a few changes because I'm afraid the current impl may try to call meta in values it shouldn't#2020-12-1800:29tony.kayok, I’m running local/root so much faster to test now#2020-12-1800:31wilkerluciooh, I think its fine, I though (meta nil) would break, but it works#2020-12-1800:32wilkerluciounless you find some issue, I'm ok to release the 2.2.31#2020-12-1800:32tony.kayyour branch name is weird#2020-12-1800:33wilkerlucioyeah... its wrong#2020-12-1800:33tony.kayso as long as the code is right, it should be ok#2020-12-1800:33tony.kayI mean version *#2020-12-1800:33wilkerlucioits fine, I'll just remove it after#2020-12-1800:33wilkerlucioand has the tag to get back to this version if needed#2020-12-1800:33tony.kayhm#2020-12-1800:34tony.kayhold on#2020-12-1800:34tony.kayI clicked around a bit#2020-12-1800:34tony.kayand other stuff seems to be slower now#2020-12-1800:34tony.kayany idea why that might be?#2020-12-1800:35tony.kaybasically it looks like similar queries that don’t have the metadata are slower now#2020-12-1800:35tony.kayby a noticeable amount#2020-12-1800:35tony.kayare you making a bunch of new atoms to fix this?#2020-12-1800:35tony.kayor rather, without the metadata would it have new overhead?#2020-12-1800:36wilkerlucionope, it should be simple#2020-12-1800:36wilkerlucioI just made another version, with an even simpler change#2020-12-1800:36wilkerluciotry the current on that branch#2020-12-1800:36wilkerluciothis are all the changes; https://github.com/wilkerlucio/pathom/compare/hotfix-final-2.3?expand=1#2020-12-1800:37wilkerluciothe previous was a bit weird: https://github.com/wilkerlucio/pathom/commit/2b1a4c368d9fdb170b31f28531608ad92d1eecce#2020-12-1800:38tony.kayok, yeah, looks good. I bounced back to .31 and that page was just as slow, so false alarm#2020-12-1800:48wilkerluciohey, I just did a comparison between 2.2 and 2.3 processing a 10k size list#2020-12-1800:48wilkerlucio
(ns com.wsscode.demos.perf
  (:require [com.wsscode.pathom.connect :as pc]
            [criterium.core :as c]
            [com.wsscode.pathom.core :as p]))

(pc/defresolver long-list [env _]
  {::pc/output [:items]}
  {:items (mapv #(hash-map :id 1) (range 10000))})

(pc/defresolver x [env {:keys [id]}]
  {::pc/input #{:id}
   ::pc/output [:x]}
  {:x (* 10 id)})

(def registry [long-list x])

(def parser
  (p/parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/reader2
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate
     ::p/plugins [(pc/connect-plugin {::pc/register registry})
                  p/error-handler-plugin
                  p/trace-plugin]}))

(comment
  (c/with-progress-reporting
    (c/quick-bench
      (parser {} [{:items [:x]}]))))
#2020-12-1800:49wilkerlucioPathom 2.2: Execution time mean : 93.091257 µs#2020-12-1800:49wilkerlucioPathom 2.3: Execution time mean : 97.180679 µs#2020-12-1800:49wilkerlucioso aparently its not just something about the reader process, I wonder if there is something around custom plugins or anything that got affected to make such difference in your case#2020-12-1801:13tony.kayI think you need each item to have props#2020-12-1801:13tony.kaymine has 3 levels of depth, but you’re right, it could be a plugin#2020-12-1801:14tony.kayI did not actually measure real overhead#2020-12-1801:14tony.kay(of every bit)#2020-12-1801:22tony.kayshould final work ok with siblings?#2020-12-1801:22wilkerlucioah, I think I had an error on the measure, got redo#2020-12-1801:22wilkerluciowhat you mean siblings?#2020-12-1801:22tony.kayquery [{:a […]} {:other […]}]..can both siblings say final?#2020-12-1801:23tony.kayon their lists of return values#2020-12-1801:23tony.kayi.e. resolver for :a and :other#2020-12-1801:24wilkerlucioyeah, you can make any list or a map final#2020-12-1801:24tony.kayok, yeah, I think that is working…I’ve peppered it around and broke something, so was verifying I understood#2020-12-1801:38tony.kay@wilkerlucio could you push that to clojars? (at least the snapshot)?#2020-12-1801:38tony.kayI want to try on my staging server, and it won’t have local root#2020-12-1801:38wilkerluciono problem, you mean at this commit: https://github.com/wilkerlucio/pathom/compare/hotfix-final-2.3?expand=1#2020-12-1801:38wilkerluciocorrect?#2020-12-1801:38tony.kaythe last one you did#2020-12-1801:39wilkerlucioyeah
#2020-12-1801:39tony.kayI’m at 58b3d629e6ef464199a06d7fcde627c403fa9d59#2020-12-1801:39wilkerluciopushed snapshot for you to try on staging#2020-12-1801:39tony.kayDid you see phronmophobic on #fulcro getting Fulcro working with Swing (I think…maybe AWT) šŸ™‚#2020-12-1801:40tony.kaythanks#2020-12-1801:40tony.kayentertaining#2020-12-1801:41wilkerluciowow, didn't, that's cool!#2020-12-1801:41wilkerlucio(snapshot ends in -3, if you need to confirm during some pipeline)#2020-12-1801:42tony.kayyeah, not retrieving yet…clojars must be slow#2020-12-1801:46tony.kayduh…should just use a frikkin sha#2020-12-1816:17tony.kayThis seems to be a problem in dev now. If I reload all nses for dev work, it pulls in pathom, which pulls in gen, which unfortunately pulls in Fulcro 2 stuff:
#error{:cause "Could not locate fulcro/client/primitives__init.class, fulcro/client/primitives.clj or fulcro/client/primitives.cljc on classpath.",
       :via [{:type clojure.lang.Compiler$CompilerException,
              :message "Syntax error compiling at (com/wsscode/pathom/gen.cljc:1:1).",
              :data #:clojure.error{:phase :compile-syntax-check,
                                    :line 1,
                                    :column 1,
                                    :source "com/wsscode/pathom/gen.cljc"},
              :at [clojure.lang.Compiler load "Compiler.java" 7647]}
             {:type java.io.FileNotFoundException,
              :message "Could not locate fulcro/client/primitives__init.class, fulcro/client/primitives.clj or fulcro/client/primitives.cljc on classpath.",
#2020-12-1816:18tony.kaybut you didn’t change anything in this rev…#2020-12-1816:18tony.kayhm, perhaps I’ll try 2.3.0 w/guardrails disabled šŸ™‚#2020-12-1720:05wilkerluciothe reason is pathom don't know its exact data, so it has to scan anyway, doing a check for each data shape could have a similar cost, with the user declaration Pathom accepts that the user know more about it, and just skips processing#2020-12-1720:12wilkerlucio@tvaughan better to always keep the nested on the output, it doesn't affect running in any way, but improves the intelisense of pathom viz for queries (better auto-complete)#2020-12-1720:13tvaughanGood to know. Thanks @wilkerlucio#2020-12-1817:26kennyI have a ::pc/batch? resolver that throws an exception. After the exception is thrown, it appears that the resolver gets called again for each item in the list, throwing each time. This results in thousands of exceptions getting thrown for a single resolver call. Is there a way to tell the resolver to end the calls if the batch resolver throws?#2020-12-1821:04wilkerluciohello, yes, its like that, what you can do is wrap the error in a try catch, and just provide empty state for the records (as if it returned all blank), this wya Pathom wont try to process then, this may help: https://cljdoc.org/d/com.wsscode/pathom/2.3.0/api/com.wsscode.pathom.connect#batch-restore-sort#2020-12-2215:17dehliIs there a single name that captures what a defmutation and a defresolver both are? I’ve gone with connector since they live in the connect namespace but was curious if there was an ā€œofficialā€ name for them.#2020-12-2215:20jmayaalvi think they could be called Operations at least looking at this ns from pathom3 https://github.com/wilkerlucio/pathom3/blob/master/src/main/com/wsscode/pathom3/connect/operation/protocols.cljc#2020-12-2215:21dehlithanks!#2020-12-2321:04wilkerluciohttps://clojurians.slack.com/archives/C8NUSGWG6/p1608757449454700#2021-12-2715:50wilkerlucioNew documentation page on Pathom 3 docs! Learn more about how Pathom 3 cache works: https://pathom3.wsscode.com/docs/cache#2021-12-2721:02wilkerlucioPlugins just landed on Pathom 3!! New docs page: https://pathom3.wsscode.com/docs/plugins/#2021-12-2800:47wilkerlucioNew caching feature on Pathom 3, you can now use a specific cache store per resolver: https://pathom3.wsscode.com/docs/cache/#custom-cache-store-per-resolver#2021-12-2918:53Michael W@wilkerlucio I've just read the docs for pathom 3. Would the plugin system allow me to add resolvers based on a spec? For example with rest apis I am doing the same operations over and over. I'd like to take the api spec, usually in json, then generate resolvers automatically. I'm already generating quite a few functions at compile time to make it easier on myself. I just wonder if spending the next month on this would be wasted, because I don't really understand the plugins from the existing docs, and am still a beginner with clojure. My question is would the plugin architecture be the place to start, or should I write it as a library?#2021-12-2918:54wilkerluciofor that you don't need plugins, plugins are for hooking inside pathom operations as they occur, the generation of resolvers is something you can do ahead of time, and then register them, so you don't need plugins for that, makes sense?#2021-12-2918:56Michael WYes, thanks, so I write the library, and just register the generated resolvers and it should work...#2021-12-2918:57wilkerlucioyup, your library can for example take some input specs (in open api or something) and return a vector of resolvers#2021-12-2918:58wilkerluciothan the user register it, something like:#2021-12-2918:58wilkerlucio(pci/register (api-wrapper/create-resolvers some-api-spec))#2021-12-2918:59Michael WThat's perfect#2021-12-2918:59Michael WThanks again.#2021-12-2919:01wilkerluciowhat kind of specs are wrapping?#2021-12-2919:02Michael WRight now I have 6 different apis I have manually written resolvers for#2021-12-2919:04Michael WI'm wanting to add a few more, and going back over my code there is a lot of repetion. I am kinda wanting to make a generic rest library to wire in any kind of rest api. It's barely an idea at this point but it beats writing the same functions over and over.#2021-12-2919:06wilkerlucioits on my backlog to have a library to support the Open API protocol#2021-12-2919:06wilkerlucioso, if I got right, you are manually doing it per api type? (like service)#2021-12-2919:06wilkerluciohttps://www.openapis.org/#2021-12-2919:07Michael WYes all of the apis I am working with seem to have those openapi specs#2021-12-2919:08Michael WIt seems the hard part is authentication, once I have that, doing post,get,patch,etc is easy.#2021-12-2919:08wilkerlucioI also guess the naming translations can be challenging#2021-12-2919:08wilkerlucioto make sure everything is correctly identified#2021-12-2919:09wilkerlucioI suggest doing some sort of prefix (like Pathom 2 does for GraphQL) in order to enable better integration without entity name collision (when handling many different services)#2021-12-2919:13Michael Wpathom.graphql/ident->alias ??#2021-12-2920:31wilkerluciohttps://blog.wsscode.com/pathom/v2/pathom/2.2.0/graphql/fulcro.html#_keywords_and_graphql_prefixes#2021-12-3011:59Reshef MannInteresting discussion. @UAB2NMK25 - take a look at this: https://vvvvalvalval.github.io/posts/2018-07-23-datascript-as-a-lingua-franca-for-domain-modeling.html Looks like he lays an approach to what you try to achieve. Hopefully you'll find it useful.#2021-12-3117:55Michael W@U015R6SQ77Z Thanks, that is an awesome article, and gave me a few ideas#2021-12-3117:55Michael Whttps://github.com/oliyh/martian#2021-12-3117:56Michael WI found that too which looks like it does exactly what I wanted but I cannot seem to get it to work against https://swapi.dev as a test#2021-12-3104:55Gleb PosobinIs there a nice way to prevent user for querying some keywords? Say I have a resolver with input = #{:user/id} and output [:user/profile-photo] and use it internally, when resolving say :user/username -> :user/id -> :user/profile-photo , but I don't want to let a user issue queries based on :user/id , they should only be aware of :user/username . Right now I am doing a prewalk over the transaction in a p/pre-process-parser-plugin and replacing all occurrences of the keyword with :forbidden , is there a better solution?#2021-12-3113:35souenzzo
(letfn [(cleanup-query [query allowed-key?]
          (->> query
               eql/query->ast
               (eql/transduce-children (filter (comp allowed-key? :dispatch-key)))
               eql/ast->query))]
  (cleanup-query [:a
                  {:b [:c]}
                  {[:d 1] [:e]}
                  '(f {:g :h})
                  '{(i {:j :k}) [:l]}]
                 #{:a :b :c :d :e 'f 'i :l}))
https://github.com/souenzzo/eql-style-guide/issues/4 I usually do that in my http/middleware stack.
#2021-12-3123:07Chris O’DonnellI feel like I'm missing something obvious; maybe someone can help. I have this resolver:
(pc/defresolver grocery-item-listing [_ _]
  {::pc/input #{}
   ::pc/output [{:grocery-items [:item/id :item/name :item/position :item/completed-at]}]}
  (go {:grocery-items [{:item/id 1
                        :item/name "Apples"
                        :item/position 1
                        :item/completed-at nil}
                       {:item/id 2
                        :item/name "Oranges"
                        :item/position 2
                        :item/completed-at nil}]}))
This query returns the expected two items:
(let-chan [result (parser {} [{:grocery-items [:item/id :item/name]}])]
    (js/console.log {:result result}))
But if I try to make the same query nested inside an ident, I get :com.wsscode.pathom.core/not-found:
(let-chan [result (parser {} [{[:component/id :grocery-list]
                                 [{:grocery-items [:item/id :item/name]}]}])]
    (js/console.log {:result result}))
I thought making a query a join nested inside an ident just added the ident attribute as an input.
#2021-12-3123:08Chris O’DonnellOh, maybe I'm missing a reader to make that happen#2021-12-3123:09Chris O’DonnellYep, added pc/open-ident-reader to my list of readers and it works as expected.#2021-01-0219:07markaddlemanIn pathom3, is it possible to have two resolvers with different inputs that yield the same output keys? It appears that the first resolver in the register vector is the one that wins#2021-01-0220:08wilkerlucioit is possible, but there is not prioritization algorithm yet, so it tries them in order, if the first fails it tries the next one#2021-01-0220:43markaddlemanfirst one failing means it returns nil?#2021-01-0221:22souenzzoafik, both exception or nil will be considered "fail".#2021-01-0221:49markaddlemanthx#2021-01-0302:05wilkerlucionil is not a fail, but the absense of the key in the response is#2021-01-0401:58Christopher GenoveseI've got a pathom resolver (in a Fulcro RAD app) that takes has empty :input and :output that looks like
[{:widget/all-widgets [:widget/id :widget/title :widget/description]}]
My resolver returns data that appears to be a valid tree, of the form
{:widget/all-widgets [{:widget/id "foo" :widget/title "bar" :widget/description "zap"}
                      {:widget/id "goo" :widget/title "car" :widget/description "aap"}
                      ...]}
But the response to the query is empty {} for reasons that are unclear to me. Is there some apparent pathom-based problem with this that might be causing the parse to fail? Thanks!
#2021-01-0408:06jmayaalv@genovese are you sure the resolver is not throwing an error? maybe if you could show the code it would be easier to see what’s going on.#2021-01-0502:46Christopher GenoveseThanks! Here's the resolver:
(defattr all-widgets :widget/all-widgets :ref
  {ao/target     :widget/id
   ::pc/output  [{:widget/all-widgets [:widget/id :widget/a :widget/b]}]
   ::pc/resolve (fn [env _]
                  #?(:clj
                     (if-let [db (get-in env [::mongo/connections :rav :db])]
                       {:widget/all-widgets (mongo/get-widgets db)}
                       (log/error "Cannot find database for rav schema!"))))})
#2021-01-0503:19Christopher GenoveseHere I used widget/a and widget/b for the title and description properties listed earlier.#2021-01-0420:27dehli#2021-01-0500:39dvingoI am getting an exception being thrown in pathom connect when attempting a mutation join with a recursive attribute in the followup query. I put the notes in this gist: https://gist.github.com/dvingo/543cb15d907587f83e41bc07eca217c8 the resolver and query work when not used as a mutation join. I figured I'd post it here first and will open an issue if this is a bug. It's not clear to me that there's anything problematic in the calling code and in the resolver.#2021-01-0603:50grzmHello! I'm just getting started with Pathom. My immediate use is to create a mock remote for Fulcro so I can test data loading without actually hooking up a backend. So, this is in cljs. I'm seeing behavior I don't understand when using defresolver. When I use a function call as the outermost part of the body, I get a bunch of ::pc/not-found results. When the outermost part is a map, it "works". Even wrapping the map in an identity call causes me to get the ::pc/not-found results šŸ˜• Here's a gist showing my explorations: https://gist.github.com/grzm/5cc47bbe645651c70fa4bcdad02e6f52#2021-01-0603:50grzmI'm sure it's something simple as I'm new to pathom and relatively inexperienced with cljs, but I'm stumped.#2021-01-0604:57wilkerluciohello, welcome to the channel! I gave a read, I see you are using the parallel-parser with reader2, thats a bad combination, I suggest you keep with the regular parser, the parallel-parser is only good in a very specific situations, for most users its unescessary complications. if you need async, use async-parser + async-reader2 also noticed you have a typo in ::pc/output (in your code its ::pc/ouput)#2021-01-0605:17grzmNice to have another pair of eyes on the code šŸ™‚#2021-01-0605:20grzmThanks for the pointers on parallel-parser. I was cribbing from examples I found in the fulcro book code: https://github.com/fulcrologic/fulcro-developer-guide/blob/master/src/book/book/pathom.cljs#L16-L25#2021-01-0605:20grzmI'm hoping to put together some simple examples of the pieces to share for those starting out.#2021-01-0605:27wilkerluciothanks, apprecited, another page you can get some starting samples: https://blog.wsscode.com/pathom/v2/pathom/2.2.0/connect/resolvers.html#2021-01-0605:27wilkerlucioand the pathom 3 docs can be a nice read too, its more organized, and altough the interface is a bit different on the edges, all the concepts are transferable: http://pathom3.wsscode.com/#2021-01-0605:29grzmThoughts on a beginner just starting with Pathom 3, and skipping Pathom 2?#2021-01-0605:29grzm(with Fulcro, primarily, but from what I gather, they're pretty independent: I'm mostly concerned with not finding examples to learn from as I get started)#2021-01-0605:40grzmI'm also wondering about the mock-http-server expecting a parser that returns a channel. Though, it looks like I might be able to mock a remote based on https://github.com/fulcrologic/fulcro/blob/932095ba094fae1b7576f85fadd2f6b2eb168ddc/src/workspaces/com/fulcrologic/fulcro/cards/form_cards.cljs#L17-L25#2021-01-0605:41grzm(I guess these are better asked in #fulcro)#2021-01-0714:18souenzzohttps://github.com/souenzzo/eql-realworld-example-app/blob/master/src/conduit/client/rest.cljs#L17#2021-01-0906:58bbssfwiw I am using fulcro + pathom3#2021-01-0623:35dvingoFigured out my problem above while trying to recreate it - the mutation was returning idents for the subtasks when they needed to be maps... So pebcak..#2021-01-0718:37Lucas SennaHi, is anyone else having issues upgrading from Pathom 2.2.x to 2.3.0? I’m trying to upgrade a cljs project from 2.2.31 to 2.3.0 but keep getting the following error during compilation:#2021-01-0719:13wilkerluciohello šŸ™‚ I can only imagine some classpath issue, did you tried looking inside the jar to verify its contents?#2021-01-0719:39Lucas SennaI did, but everything seems fine 🤷#2021-01-0719:40Lucas Senna
āžœ jar tf pathom-2.3.0.jar
META-INF/MANIFEST.MF
META-INF/maven/com.wsscode/pathom/pom.xml
META-INF/leiningen/com.wsscode/pathom/project.clj
META-INF/leiningen/com.wsscode/pathom/README.md
META-INF/leiningen/com.wsscode/pathom/LICENSE
META-INF/
META-INF/maven/
META-INF/maven/com.wsscode/
META-INF/maven/com.wsscode/pathom/
META-INF/maven/com.wsscode/pathom/pom.properties
union_resolver_test.clj
com/
com/wsscode/
com/wsscode/pathom/
com/wsscode/pathom/parser.cljc
com/wsscode/pathom/merge.cljc
com/wsscode/pathom/map_db.cljc
com/wsscode/pathom/profile.cljc
com/wsscode/pathom/gen.cljc
com/wsscode/pathom/diplomat/
com/wsscode/pathom/diplomat/http/
com/wsscode/pathom/diplomat/http/fetch.cljs
com/wsscode/pathom/diplomat/http/clj_http.clj
com/wsscode/pathom/diplomat/http.cljc
com/wsscode/pathom/graphql.cljc
com/wsscode/pathom/specs/
com/wsscode/pathom/specs/ast.cljc
com/wsscode/pathom/specs/query.cljc
com/wsscode/pathom/core.cljc
com/wsscode/pathom/fulcro/
com/wsscode/pathom/fulcro/network.cljs
com/wsscode/pathom/misc.cljc
com/wsscode/pathom/test.cljc
com/wsscode/pathom/sugar.cljc
com/wsscode/pathom/trace.cljc
com/wsscode/pathom/connect.cljc
com/wsscode/pathom/connect/
com/wsscode/pathom/connect/gen.cljc
com/wsscode/pathom/connect/graphql.cljc
com/wsscode/pathom/connect/indexes.cljc
com/wsscode/pathom/connect/planner.cljc
com/wsscode/pathom/connect/test.cljc
com/wsscode/pathom/connect/foreign.cljc
com/wsscode/pathom/connect/graphql2.cljc
com/wsscode/common/
com/wsscode/common/async_cljs.cljs
com/wsscode/common/combinatorics.cljc
com/wsscode/common/async_clj.clj
com/wsscode/common/async_cljs.clj
com/wsscode/demos/
com/wsscode/demos/meta.clj
#2021-01-0719:43Lucas Senna2 of my colleagues are facing the same issue, so I don’t think it is a problem with my machine. Could it be something with the way that Shuffle is configured?#2021-01-0720:42wilkerlucioI just tried to reproduce in a full setup, both on CLJ or CLJS (using Shadow), but here worked fine in both cases#2021-01-0720:43wilkerlucioI deleted the libraries to force a re-download before, but still fine#2021-01-0720:43wilkerluciocan you make a reproduction repo?#2021-01-0720:43wilkerluciois it working on other machines you know?#2021-01-0720:44wilkerlucioI also suggest checking version of core.async, just in case#2021-01-0723:29Lucas Sennaloading#2021-01-0823:24Lucas SennaBumping org.clojure/core.async to the latest stable version did the trick. Thanks a lot, man.#2021-01-0720:45wilkerluciohello everybody, happy new year! I'm glad to announce a very long wanted feature in Pathom, now in Pathom 3 you can do resolvers with nested inputs! check latest on github, docs on nested inputs: https://pathom3.wsscode.com/docs/resolvers/#nested-inputs.#2021-01-0720:47wilkerluciothis required considerable changes, including changes in the indexes of Pathom, I'll give more details in the next blog post I'll make. for now if you are curious these were the changes involved in it: https://github.com/wilkerlucio/pathom3-docs/pull/5/files#2021-01-0802:27wilkerlucioand to start the good year, another processing feature: now Pathom 3 supports optional inputs! https://pathom3.wsscode.com/docs/resolvers/#optional-inputs#2021-01-0811:40souenzzo@wilkerlucio in situations like
;; resolver 1
{:input [:a :b]
 :ouput [:c]}
;; resolver 2
{:input [:a]
 :ouput [:c]}
;; entity
{:a 1 :b 2}
Will resolver 1 *always* be called? If possible, answer both in pathom2 and pathom3 😜
#2021-01-0812:08wilkerlucioquick answer: no#2021-01-0812:08wilkerlucioin pathom2 that depends on the resolver weight, whatever Pathom things is the faster path, it will take it#2021-01-0812:09wilkerlucioin pathom3 there still no prioritization yet (altough that's the next thing I'll be working on), so in Pathom 3 its kind random, but will always be same, for now#2021-01-0818:43wilkerluciowill be merged soon#2021-01-0818:43wilkerlucio(in this value2 is always tried first, default priority is 0, higher numbers go first)#2021-01-0818:44wilkerluciook, I think I'm getting somewhere, this is a way to force one path to be preferred over another:
(is (= (run-graph
                   (pci/register
                     [(pco/resolver `value
                        {::pco/input  [:a :b]
                         ::pco/output [:value]}
                        (fn [_ _]
                          {:value 1}))
                      (pco/resolver `value2
                        {::pco/input    [:c]
                         ::pco/output   [:value]
                         ::pcr/priority 1}
                        (fn [_ _]
                          {:value 2}))
                      (pbir/constant-resolver :a 1)
                      (pbir/constant-resolver :b 2)
                      (pbir/constant-resolver :c 3)])
                   [:value]
                   {})
                 {:c     3
                  :value 2}))
#2021-01-0815:13jmayaalv@wilkerlucio will there be an alternative for :com.wsscode.pathom.core/not-found on pathom3? starting to migrate a green field app to pathom3 and we are really happy with the simplicity šŸ™‚#2021-01-0815:15wilkerlucio@jmayaalv the alternative is check if the key is in the response, in case of not-found, the attribute is gonna be absent, I guess that can solve for most cases, if not, troubleshooting will come as an extension of the new "run stats" thing, hard to explain in brief, but there you can find most data needed to figure any situation, you can take a look at that by checking the meta on the responses (for EQL interface)#2021-01-0815:23wilkerlucioimportant thing to take note: each response map has its on run stats (each is a different run context), so if you have a response like {:foo {:bar "baz"}}, there are 2 run stats, one for the outermost map, and one for the inner, in cases of collections, each entry has its own stats#2021-01-0815:24jmayaalvi will check the plugins out, although absent of value should be good enough for us. Thank you !#2021-01-0815:42imreIs there any way for a resolver to set a parameter on one of its outputs?#2021-01-0815:47wilkerluciowhat you mean set a parameter on its outputs?#2021-01-0815:47imrejust trying to come up with a proper example šŸ™‚#2021-01-0815:56imreso, assuming I have the following resolver:#2021-01-0815:56imre
(pc/defresolver output-resolver
  [env {value :foo/intermediate}]
  {::pc/input  #{:foo/intermediate}
   ::pc/output [:foo/output]}
  {:foo/output [value (-> env :ast :params :some-param)]})
#2021-01-0815:56imreI can use the query
[{[:foo/intermediate 123]
  [(:foo/output {:some-param "value"})]}]
#2021-01-0815:56imreand get {:foo/output [123 "value"]} back#2021-01-0815:57imre(not guaranteed to be correct, I'm just typing this into a text editor)#2021-01-0815:57imreI want to create an intermediate resolver, like
(pc/defresolver intermediate-resolver
  [_ {value :foo/input}]
  {::pc/input  #{:foo/input}
   ::pc/output [:foo/intermediate]}
  {:foo/intermediate value})
#2021-01-0815:58imreAnd within this resolver I want to determine a value for :some-param and pass it on to output-resolver#2021-01-0816:00imreso a query of `[{[:foo/input 123] [:foo/output]}]` could return {:foo/output [123 "value-set-by-intermediate-resolver"]}#2021-01-0816:02wilkerluciono, you can't change things like this sideways, if you need information to flow, add new attributes to the system, make the output of the resolver respond with more keys, and use those keys in the input forward, makes sense?#2021-01-0816:24imreIt does I guess, I primarily just wanted to find out whether it was possible. So for example, if there's a resolver that supports pagination via a parameter, and some other resolver wants to call it with pagination, then pagination will need to be made an input. Is that correct?#2021-01-0816:27csgerofor context, this came up while experimenting with foreign-resolvers, and the second resolver is actually implemented in a different service#2021-01-0816:35wilkerluciook, foreign is still a very exploratory area, but I would try to avoid going too deep on this idea, since from the Pathom perpective that is hard to happen, one way to think of it is that params are always a "local" thing, while inputs are things that flow across#2021-01-0816:35imreThe concept of parameters is interesting in that it's orthogonal to outputs. However, if they cannot be built on with other resolvers, that could make it risky to depend on them#2021-01-0816:35wilkerlucioone acceptable thing is for a resolver to accept some data as input OR params, this way you can have both options#2021-01-0816:36wilkerluciothe just released optional inputs on Pathom 3 can help with that, so you can have soft dependencies on specific attributes#2021-01-0816:37wilkerluciobut about foreign, its still experimental area, more work on it will come in Pathom 3, and feedback is very appreciated, if you have some cases you like to share, I'll love to hear about them#2021-01-0816:38wilkerlucioanother alternative to sideways communication is to have some mutable state in the env, you can add a new atom there, modify from one resolver and read in the other, this is a tricky option and I suggest a lot of care if you try this path, but its there#2021-01-0817:38imreThank you for the insight. Fortunately in this case the foreign service is under control so we can change it. Will be a bit of extra work but I certainly wouldn't want to use the lib in a non-intended way. I'm sure we'll get back to you wrt foreign resolvers šŸ˜‰#2021-01-0816:28Ian FernandezHas anyone here used Pathom as a Dependency Injection framework like @souenzzo did here? https://github.com/souenzzo/souenzzo.github.io/blob/master/conduit/src/br/com/souenzzo/conduit/ssr.clj#L270#2021-01-0816:28Ian FernandezAny thoughts about using it? (pathom2 examples could be interesting too šŸ™‚ )#2021-01-0816:29Ian FernandezI'm looking for a db component + http component service just to example šŸ˜„#2021-01-0816:29Ian FernandezI'll do some comparison between Integrant and this approach#2021-01-0816:29Ian FernandezI was looking for some tips about this#2021-01-0816:33wilkerlucio@d.ian.b dependency injection in Pathom is accomplished by env, you can add things to env at call time (when calling the parser), or in any point you want, so you can pull those in any part of the system, makes sense?#2021-01-0817:03Ian FernandezI will try doing some comparison between Integrant / Pathom then šŸ˜„#2021-01-0817:25myguidingstarimo dependency injection is quite easy in Clojure because of immutable data structure. You can use Pathom, Prismatic Plumbing, Integrant or even Datascript. The (harder) related problem is starting/stopping components and that's what Integrant (and Duct) does#2021-01-0817:26Ian Fernandezwhy not start / stopping components via pathom?#2021-01-0817:58wilkerlucioyou can, and for that I suggest you can use the https://pathom3.wsscode.com/docs/cache#custom-cache-store-per-resolver to share a cache on the resolvers that are for building up initialization#2021-01-0817:58wilkerluciothis way you can make them persistent, at the same time they will delay initialization until something in the system needs them#2021-01-0817:59wilkerlucio@d.ian.b if you want I can review the code with you once you have it#2021-01-0817:59wilkerlucioaltough, there is no stop šŸ˜›#2021-01-0818:00Ian Fernandezand if I want to use pathom2 for this?#2021-01-0818:01wilkerlucioagain, altough you can, I think something like "component" still a better option, because on Pathom you can't stop in reverse order, so not as good to this kind of initialization#2021-01-0817:00royalaidHey @wilkerlucio I kinda did some cursory poking around and couldn’t find anything analogous to async parsing in pathom2 so that I can use core.async with pathom3. Any pointers?#2021-01-0817:01wilkerlucioits on the backlog, but not available in Pathom 3 yet#2021-01-0817:01wilkerluciobut I hope an initial version of it should land in the next few weeks#2021-01-0818:44wilkerluciook, I think I'm getting somewhere, this is a way to force one path to be preferred over another:
(is (= (run-graph
                   (pci/register
                     [(pco/resolver `value
                        {::pco/input  [:a :b]
                         ::pco/output [:value]}
                        (fn [_ _]
                          {:value 1}))
                      (pco/resolver `value2
                        {::pco/input    [:c]
                         ::pco/output   [:value]
                         ::pcr/priority 1}
                        (fn [_ _]
                          {:value 2}))
                      (pbir/constant-resolver :a 1)
                      (pbir/constant-resolver :b 2)
                      (pbir/constant-resolver :c 3)])
                   [:value]
                   {})
                 {:c     3
                  :value 2}))
#2021-01-0820:01kendall.buchananDoes anyone know where an example is in the documentation (or anywhere, really) about how to construct a query for a resolver with two items in ::pc/input?#2021-01-0820:01kendall.buchanane.g. ::pc/input #{:item-1 :item-2}#2021-01-0820:18kendall.buchananhttps://blog.wsscode.com/pathom/#_multiple_inputs indicates that it can be done, as the ā€œinput to a resolver is a setā€, but I can’t find any examples anywhere.#2021-01-0820:19wilkerlucio@kendall.buchanan the example you sent is correct, are you having troubles to make it work?#2021-01-0820:22kendall.buchananI just don’t know what a query looks like that resolves to that resolver.#2021-01-0820:23kendall.buchanan[{[[:item-1 "hello"] [:item-2 "world"]] [:what-im-fetching]}]#2021-01-0820:23kendall.buchananThis doesn’t appear to work.#2021-01-0820:25kendall.buchananThe best I can come up with is to combine them into a single ::pc/input #{:item-1+2} with a query like [{[:item-1+2 {:item-1 "hello" :item-2 "world"] [:what-im-fetching]}]#2021-01-0820:28kendall.buchananGood to hear from you @wilkerlucio, by the way—we had dinner together a couple years ago at Clojure/conj šŸ‘‹. One of our devs has jumped all in with Pathom; I’m just getting my feet wet.#2021-01-0820:35wilkerlucioglad to hear about you too, great times at Conj, can't wait for us to be able to do another one of those again šŸ™‚#2021-01-0820:34wilkerlucio@kendall.buchanan you can find an example here: https://blog.wsscode.com/pathom/v2/pathom/2.2.0/connect/resolvers.html#_multiple_inputs#2021-01-0820:34wilkerluciobut other than a query itself, you could start from one data point, then a resolver realizes those 2 attributes, which then calls the next one#2021-01-0821:06kendall.buchananOkay, I’m wondering if I’m genuinely not understanding…#2021-01-0821:06kendall.buchanan`[{([:customer/id 123] {:pathom/context {:customer/first-name ā€œFooā€ :customer/last-name ā€œBarā€}}) [:customer/full-name]}]`#2021-01-0821:06wilkerluciolets go by steps šŸ™‚#2021-01-0821:07wilkerluciothe [:customer/id 123] will set the :customer/id value, and the :pathom-context (when sent as a param to the ident) merges that data in as well#2021-01-0821:07kendall.buchanan::pc/input #{:customer/id :customer/first-name :customer/last-name}#2021-01-0821:08wilkerlucioyou probably don't need those 3, you could use any ident there#2021-01-0821:08wilkerlucioPathom is about "shape matching", so imagine that if you have enough to trigger a resolver, pathom will do it#2021-01-0821:08kendall.buchananSure…#2021-01-0821:08kendall.buchananBut some data requires more than one input.#2021-01-0821:09kendall.buchananHere’s a more concrete example…#2021-01-0821:09kendall.buchananI have a user with multiple logins across multiple apps.#2021-01-0821:09kendall.buchananTrying to fetch a record of the most recent login.#2021-01-0821:10kendall.buchananI can build a chain of resolvers that will eventually get me that information, from a single user ID…#2021-01-0821:10kendall.buchananBut that inevitably passes through many more layers than is necessary.#2021-01-0821:10kendall.buchananAssuming I can pass a user ID, and an app ID.#2021-01-0821:10kendall.buchanan(This is not my exact situation, but it simplifies it.)#2021-01-0821:12wilkerluciohere is a full example:#2021-01-0821:12wilkerlucio
(ns com.wsscode.demos.multiple-inputs
  (:require [com.wsscode.pathom.core :as p]
            [com.wsscode.pathom.connect :as pc]))

(def users-db
  {1 {:user/first-name "Sam"
      :user/last-name  "Rock"}})

(pc/defresolver user-by-id [env {:keys [user/id]}]
  {::pc/input  #{:user/id}
   ::pc/output [:user/first-name :user/last-name]}
  (get users-db id))

(pc/defresolver full-name [env {:user/keys [first-name last-name]}]
  {::pc/input  #{:user/first-name :user/last-name}
   ::pc/output [:user/full-name]}
  {:user/full-name (str first-name " " last-name)})

(def registry
  [user-by-id full-name])

(def parser
  (p/parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/reader2
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate
     ::p/plugins [(pc/connect-plugin {::pc/register registry})
                  p/error-handler-plugin
                  p/trace-plugin]}))

(comment
  ; just use the ident, pull name from resolver
  (parser {} '[{[:user/id 1] [:user/full-name]}])

  ; provide data, note we use an id that dont even exist
  (parser {} '[{([:user/id 3] {:pathom/context {:user/first-name "Foo" :user/last-name "Bar"}})
                [:user/full-name]}])

  ; to show the ident doesn't matter, changing it for anything else
  (parser {} '[{([:whatever-we-want "bla"] {:pathom/context {:user/first-name "Foo" :user/last-name "Bar"}})
                [:user/full-name]}]))
#2021-01-0821:12wilkerluciothe :user/full-name does what you said, depending on multiple attributes, and getting then from the fake db load#2021-01-0821:14kendall.buchananBut what if this person has multiple names he goes by. You know what I mean?#2021-01-0821:14kendall.buchananIn this example, the first and last name are derived automatically.#2021-01-0821:16wilkerlucionot sure if I get what you mean#2021-01-0821:16wilkerluciowhat you mean multiple names?#2021-01-0821:17kendall.buchanan
(def users-db
  {1 {:public-name {:user/first-name "Sam"
                    :user/last-name  "Rock"}
      :secret-name {:user/first-name "Ram"
                    :user/last-name  "Sock"}}})
#2021-01-0821:18wilkerlucioin the core its about attribute relations, and how one attribute relates to another, so in this case we are expressing what a full name means#2021-01-0821:18wilkerluciothat example you just sent is a different thing#2021-01-0821:18kendall.buchananThat’s what I’m trying to get at: how do I get to my data assuming there is no derivation for the second piece of data.#2021-01-0821:19wilkerlucioits not just multiple inputs, but nested inputs, when you need to depend on some deeper shape, you would have to make a decision of which to pick in the end somehow#2021-01-0821:19kendall.buchananThat the query is fundamentally incomplete without two data points.#2021-01-0821:19wilkerlucioI woulnd't say incomplete, it just depends on your system#2021-01-0821:19wilkerluciothat example I show on full name is a common one, and in most systems the user will only have one field for it#2021-01-0821:20wilkerluciothen its about modeling the semantics of your system#2021-01-0821:20wilkerluciothe nested thing you sent is possible in Pathom 3: https://pathom3.wsscode.com/docs/resolvers#nested-inputs#2021-01-0821:20wilkerluciobut I feel like this is not what you thinking about, I fell like you have some misconception we didn't figure here yet#2021-01-0821:21kendall.buchananI see where you’re going about nested resolvers…#2021-01-0821:21kendall.buchananOr subqueries, I suppose.#2021-01-0821:21kendall.buchananBut, why is a subquery necessary if I already have my data?#2021-01-0821:21kendall.buchananIf I know X and Y, why can I not write a resolver that reacts to the existence of X and Y?#2021-01-0821:21wilkerluciosometimes you dont#2021-01-0821:21wilkerlucioas the aggrgation example I show in the docs#2021-01-0821:21wilkerlucioyou can#2021-01-0821:21wilkerlucioI still dont understand what you are trying to do#2021-01-0821:22kendall.buchananTake the example you posted here…#2021-01-0821:22wilkerluciowhat is the shape you have? what do you want in the end?#2021-01-0821:22kendall.buchananAssume I’m looking up a name, based on an ID.#2021-01-0821:22kendall.buchanan(Not caring for :customer/full-name, just looking up a name.)#2021-01-0821:22wilkerluciook#2021-01-0821:22kendall.buchananBut the user has multiple names, of different types.#2021-01-0821:22wilkerluciolets give names, you want :user/name, you have :user/id#2021-01-0821:23kendall.buchananNo… not quite…#2021-01-0821:23wilkerluciowhat you mean multiple names? are from multiple sources?#2021-01-0821:23wilkerluciocan you give an example?#2021-01-0821:23kendall.buchananYeah, lemme write out some data…#2021-01-0821:25wilkerlucioits helps to think about the output shape you want, so we can build the chain from it, and in the end you have to give names to every attribute you want to return, thinking like that often helps me figure it out#2021-01-0821:25kendall.buchananYou know what, I think you’re right: it’s about the semantics of the keys.#2021-01-0821:25kendall.buchananRight…#2021-01-0821:25kendall.buchananI’m imagining my resolver working with this:#2021-01-0821:25kendall.buchanan
(def users-db
  {1 {:public-name {:user/first-name "Sam"
                    :user/last-name  "Rock"}
      :secret-name {:user/first-name "Ram"
                    :user/last-name  "Sock"}}})
#2021-01-0821:26kendall.buchananWhen, I think what you’re saying is…#2021-01-0821:26kendall.buchanan
(def users-db
  {1 {:user.public/first-name "Sam"
      :user.secret/first-name "Ram"}})
#2021-01-0821:26wilkerlucioyup#2021-01-0821:26kendall.buchananIn other words…#2021-01-0821:26wilkerlucioin Pathom its good to keep things as flat as possible#2021-01-0821:26kendall.buchananIt pushes the burden of shaping the data further down into the system.#2021-01-0821:26wilkerluciogives more leverage#2021-01-0821:26kendall.buchanan(In my real world example.)#2021-01-0821:27kendall.buchananBut how would that work when user.x is unknown/#2021-01-0821:27kendall.buchanan?#2021-01-0821:27kendall.buchanan:user.x/first-name#2021-01-0821:28kendall.buchananOr infinite options.#2021-01-0821:28wilkerlucioyou can't do that, Pathom must know about the attributes of the system, infinite attributes is not a thing#2021-01-0821:28kendall.buchananI’ll have to think on that.#2021-01-0821:28wilkerluciobecause Pathom is based on index processing, it has to know ahead of time about all relationships#2021-01-0821:28wilkerlucioI'm curious where you land, IME this has never been a limitation so far, maybe you find one, hehe#2021-01-0821:29kendall.buchananThe situation I have is this…#2021-01-0821:29kendall.buchananWe have an infinite number of users (with ids)…#2021-01-0821:29kendall.buchananAnd they play ā€œgamesā€, I suppose you could call them.#2021-01-0821:30kendall.buchananAnd the number of games they play is infinite.#2021-01-0821:30kendall.buchananBut the number of times they play the game is also infinite.#2021-01-0821:30kendall.buchananSo, I’m trying to fetch a record of their most recent session, given a user ID, and a game ID.#2021-01-0821:30kendall.buchananSimple SQL query.#2021-01-0821:30kendall.buchananWHERE clause takes two params.#2021-01-0821:31kendall.buchananBut I can’t understand how this translates to Pathom.#2021-01-0821:31kendall.buchananTakes two joins in SQL across three tables. It’s a junction, essentially.#2021-01-0821:31kendall.buchananAnd I don’t see how Pathom handles a junction—where two indexes are required to resolve it.#2021-01-0821:32kendall.buchanan(Thank you for all this attention, by the way—surely you have important things to be doing.)#2021-01-0821:32wilkerlucioit really helps if you start naming the attributes that participates in this query, I dont see any unbounded attributes in what you said, from the messages I can extract: :user/id :game/id, :session/id (maybe), I mean, Pathom is a connector, you can make the resolver do anything#2021-01-0821:32wilkerluciowhat you described looks like a resolver with a query#2021-01-0821:32wilkerluciothe session is another table?#2021-01-0821:33kendall.buchananThe ā€œclientā€ has the :user/id and :game/id, and is seeking a :session/id.#2021-01-0821:33wilkerluciowhat you want to get select * from game_session where userid = 10 AND gameid= 20, like this?#2021-01-0821:33wilkerlucioand then, from the session, what data you expect out (name in attributes)#2021-01-0821:33kendall.buchananYeah, more or less.#2021-01-0821:33wilkerlucio?#2021-01-0821:33kendall.buchanan:session/id.#2021-01-0821:33kendall.buchananThat is what’s needed.#2021-01-0821:35wilkerlucio
(pc/defresolver user-session [env {user-id :user/id game-id :game/id}]
  {::pc/input  #{:user/id :game/id}
   ::pc/output [:session/id]}
  (let [res (fake-sql (str "select session_id from sessions where user_id=" user-id " and game_id = " game-id))]
    {:session/id (:session_id res)}))
#2021-01-0821:35wilkerluciosomething like this#2021-01-0821:35wilkerlucioyou probably want a different resolver to generate those combinations#2021-01-0821:35wilkerluciosomething like:#2021-01-0821:36kendall.buchananYeah. My original question was how to construct the EQL for that: ::pc/input #{:user/id :game/id}#2021-01-0821:37wilkerlucio
(pc/defresolver games-from-user [env {:keys [user/id]}]
  {::pc/input  #{:user/id}
   ::pc/output [{:user/games [:user/id :game/id]}]}
  ; assume this returns a list of game ids
  (let [res (find-games-from-user env id)]
    {:user/games (mapv #(hash-map :user/id id :game/id %) res)}))
#2021-01-0821:37wilkerlucionow you can run: [{[:user/id 1] [{:user/games [:session/id]}]}]#2021-01-0821:38wilkerlucioits really up to you to model, its just different, and takes some practice to adjust#2021-01-0821:38kendall.buchananYeah. IMO, this is constraint that I don’t understand.#2021-01-0821:38kendall.buchananTwo constraints it imposes…#2021-01-0821:39kendall.buchananPrevents the resolver writer from tailoring more performant resolvers.#2021-01-0821:39wilkerlucioI suggest you take a read on this page: https://pathom3.wsscode.com/docs/planner#2021-01-0821:39kendall.buchananAnd limits the type granularity of responses for the client.#2021-01-0821:39wilkerluciothis explains how pathom 3 plans for the graph, pathom 2 is bit different, but the main idea is the same, when it sees something the user wants, it traverses the indexes for that attribute, to figure how to reach it#2021-01-0821:40kendall.buchananBut if I have a SQL query that can fetch one item faster than N, your example suggests it can’t be used.#2021-01-0821:40wilkerlucioyou can create as many paths as you want, so you can make a direct one if you want to#2021-01-0821:40wilkerlucioand we are limiting our chat here around "static resolvers"#2021-01-0821:41wilkerlucioto get as efficient as what you saying, there are the dynamic resolvers (still getting mature), but their ability is to have dynamic inputs and outputs, that responds depending on the user requirement (and dependencies requirements)#2021-01-0821:41wilkerluciothis is done to call GraphQL for example, where it would be very inneficient if each attribute made its own call#2021-01-0821:41kendall.buchananIn other words…#2021-01-0821:41kendall.buchanan::pc/input #{:user/id :game/id}#2021-01-0821:41wilkerlucioso the dynamic resolvers have a different thing, they can optimize laterally#2021-01-0821:41kendall.buchananThere is no way to satisfy this, correct?#2021-01-0821:42wilkerluciowhat you mean you can't satisfy?#2021-01-0821:42kendall.buchananIn a single trip to the resolver index.#2021-01-0821:42wilkerlucioyou can provide the data for it, or provide resolvers to reach it#2021-01-0821:42kendall.buchananyou can provide the data for it#2021-01-0821:42wilkerluciohave you seen what the indexes look like?#2021-01-0821:42kendall.buchananWhat does the query look like?#2021-01-0821:43wilkerlucioI sent you at the two final examples:
(comment
  ; just use the ident, pull name from resolver
  (parser {} '[{[:user/id 1] [:user/full-name]}])

  ; provide data, note we use an id that dont even exist
  (parser {} '[{([:user/id 3] {:pathom/context {:user/first-name "Foo" :user/last-name "Bar"}})
                [:user/full-name]}])

  ; to show the ident doesn't matter, changing it for anything else
  (parser {} '[{([:whatever-we-want "bla"] {:pathom/context {:user/first-name "Foo" :user/last-name "Bar"}})
                [:user/full-name]}]))
#2021-01-0821:43wilkerluciousing :pathom/context param is a way#2021-01-0821:43wilkerlucioyou can also provide though the env, setting ::p/entity (atom {:data "here"}#2021-01-0821:43kendall.buchananAnd either key can be in the tuple?#2021-01-0821:43wilkerlucioyeah, the key there doesn't matter#2021-01-0821:44wilkerluciothis is more a legacy from Fulcro integration, in this case (for Fulcro) the ident part can tell which local table it should use, without matter what the rest of the context is#2021-01-0821:44wilkerluciothis an example providing data via env:#2021-01-0821:44wilkerlucio
(parser {::p/entity (atom {:user/first-name "Foo" :user/last-name "Bar"})}
    '[:user/full-name])
#2021-01-0821:44kendall.buchanan([:user/id 3] {:pathom/context {:game/id 1}}]) is the same as ([:game/id 1] {:pathom/context {:user/id 3}}])#2021-01-0821:44wilkerlucioyeah#2021-01-0821:44kendall.buchananOkay. I’m assuming that was never desirable, right?#2021-01-0821:45wilkerlucioit only shapes your output shape (which will match the ident)#2021-01-0821:45kendall.buchananJust… legacy, like you say.#2021-01-0821:45wilkerlucioits was a later addition to support extra inputs at query level#2021-01-0821:45kendall.buchananK, I’ll give all your feedback some thought for how I construct our actual resolvers. Thank you again. Big help. Love the project.#2021-01-0821:46kendall.buchananIf I had any feedback, it seems to me that multiple params in the queries ought to be natively supported, but perhaps I just have more adjusting to do.
#2021-01-0821:46wilkerlucioin Pathom 3 you can use placeholders for a simpler version of it#2021-01-0821:47kendall.buchananYou feel like it’s ready for real use?#2021-01-0821:47wilkerlucionot yet, but its quite easy to implement that feature in Pathom 2, I think I'll do that today, because you are not the first person asking, hehe#2021-01-0821:47kendall.buchananOh good, hehe. Thank you !#2021-01-0821:48wilkerlucioyou can see what it looks like here: https://pathom3.wsscode.com/docs/placeholders#provide-data#2021-01-0821:49wilkerlucioPathom 3 is getting there, I think its good for experiments and small usages, but its lacking tooling, so when things start to get complicated, the tooling can be quite missed#2021-01-0821:50wilkerluciobut in terms of core features, it has a lot done already#2021-01-0821:06wilkerluciohello everyone, another feature out šŸŽ‰! Now you can express different priorities for resolvers, so in cases that pathom need to choose, you can have a say on it: https://pathom3.wsscode.com/docs/resolvers#prioritization#2021-01-0917:28wilkerlucio#2021-01-1022:53alex-ebertsI’m building a simple Todo app in Fulcro where I have a report that lists projects. Projects can have many Todos (via a ref attribute). My report’s source-attribute is :projects/all-projects, a global resolver that returns all :project/ids. My report columns are ā€œ:project/labelā€ and ā€œ:todo/labelā€ (:todo/label is a ā€œto-manyā€ nested connection of :project/id). When I render the report, it displays the :project/label and a vector of id’s in place of the :todo/label. What’s the proper way to have the report render the individual :todo/labels themselves instead of the ā€œidents in a listā€ that it displays by default? (My apologies if this is not the right channel for this question…)#2021-01-1114:06dehlican you include what your current query looks like?#2021-01-1115:35alex-ebertsHi @U2U78HT5G - the ā€œsource-attributeā€ is the entry point for the query in a Fulcro RAD report. My source-attribute is:
(defattr all-projects :project/all-projects :ref
  {ao/target     :project/id
   ao/pc-output  [{:project/all-projects [:project/id]}]
   ao/pc-resolve (fn [{:keys [query-params] :as env} _]
                   #?(:clj
                      {:project/all-projects (queries/get-all-projects env query-params)}))})
It returns a list of :project/ids. The other resolvers in the system can reach the report columns: :project/label and :project/project-todos. Here’s a link to the code: https://github.com/aeberts/fulcro-rad-demo/blob/f0dbc0129068db97e28a5bcd88defc3a7bdacf98/src/shared/com/example/ui/project.cljc#L32
#2021-01-1113:00MatthewLispHello šŸ‘‹#2021-01-1120:59ChicĆ£oHello!#2021-01-1116:57wilkerlucioNew docs section on using Recursive queries with EQL on Pathom 3: https://pathom3.wsscode.com/docs/eql/#recursive-queries#2021-01-1121:31markaddlemanHave you given any thought to multimethod-like dispatch for resolvers? For example, I might have a compute-cost resolver that dispatches based on an input parameter product/type to resolvers compute-book-cost and compute-fish-cost that have different inputs. I can see how I might use optional inputs to achieve the desired result but that approach seems a little messy.#2021-01-1122:46wilkerlucionot sure if I understand the issue, you can already have multiple options for the same value (multiple resolvers with same output, different input), with different dependencies. can you make an example?#2021-01-1122:56markaddlemanI have a use case of two resolvers taking different inputs and producing the same output.#2021-01-1122:57markaddlemanWhen the inputs for resolver A are satisfied, I'd like Pathom to execute that resolver. When the inputs for resolver B are satisfied, obviously execute the other.#2021-01-1122:59markaddlemanI can see a way to implement this in Pathom 3 using optional inputs but it seems a little messy.#2021-01-1123:29wilkerlucioyou don't need optional input, just make 2 resolvers with the same output, the exact thing you described is what pathom does#2021-01-1123:32wilkerluciocode in Pathom 3 (but works the same in Pathom 2):#2021-01-1123:32wilkerlucio
(pco/defresolver c-from-a [{:keys [a]}]
  {:c (str a "A")})

(pco/defresolver c-from-b [{:keys [b]}]
  {:c (str b "B")})

(def paths-env (pci/register [c-from-a c-from-b]))

(p.eql/process paths-env {:a 1} [:c])
=> {:c "1A"}
(p.eql/process paths-env {:b 2} [:c])
=> {:c "2B"}
#2021-01-1218:03wilkerluciohello folks, if you are looking to learn more on Pathom 3, I just released a new tutorial, in it you can learn how to use Pathom 3 to scrape data out of Hacker News šŸŽ‰! this is a medium/large tutorial and goes over many different pieces of pathom 3 (modeling, optional inputs, recursive queries...), check it out at: https://pathom3.wsscode.com/docs/tutorials/hacker-news-scraper#2021-01-1219:31royalaidSo right off the bat this will be probably be an entry point for a lot of people into pathom3 so it might be smart to make this "blog post" like and some motivation and high level overview to the top of the fold#2021-01-1219:32royalaidI also am aware that this is a page nested on the docs site so it might not make sense but I feel making a good first impression is super important#2021-01-1221:05wilkerlucioI think makes sense, you mean like more background on pathom and why to use for this task?#2021-01-1312:47royalaidYeah, a simple bit of "this is pathom, and this why it is awesome" at the top would go a long way#2021-01-1412:38Vincent CantinOut of curiosity, when using a bounded recursive query, is there a way to know if the query's result was bounded ?#2021-01-1415:12wilkerlucionot really, you can try to infer from the results, but you can't be sure if its done because there is no more data, or if was capped by Pathom#2021-01-1415:12wilkerluciodo you have any use cases in mind for it?#2021-01-1415:29Vincent CantinNo use case, just wondering.#2021-01-1519:31wilkerluciohello everybody, I want to share with you some findings I just did. I'm working on the async support for Pathom 3, and this time I decided to compare two different implementations, one based in core.async and one based in promesa. I made the two implementations, and ran some benchmarks, here are the results: https://gist.github.com/wilkerlucio/a1cda78802c8a5f9cab92cb7c52fa69b#2021-01-1519:32wilkerlucio#2021-01-1519:32wilkerluciowhen I tried the core.async implementation I was a bit sad with the performance results, but than seeing those with Promesa made me really excited! the overhead using Promesa is quite close to the serial processing, which is a lot faster than every version of Pathom before that! and I would love to hear if have any opinions about this direction šŸ™#2021-01-1521:01imreImpressive! Can it be made pluggable?#2021-01-1521:20wilkerlucioyou can convert anything to a promise-like from promise, as long as you have some way to trigger the callbacks (for resolving or rejecting), so you could wrap a channel to make it do it. is this what you were asking about?#2021-01-1522:22dehliit would be great if there could be one spot that we could do the channel -> promise conversion so that we could have our resolvers just return channels if we wanted#2021-01-1522:24dehliand then we can appropriately handle promise rejections in one spot as well since traditionally core async channels don’t really have a notion of ā€œrejectingā€#2021-01-1523:24dehliSomething similar to how sieppari works would be awesome where by default it only supports promesa but you can bring in another namespace to extend what async libraries are supported (https://github.com/metosin/sieppari#external-async-libraries)#2021-01-1608:20imreThis ^#2021-01-1616:12wilkerlucioI did some play here, and I checked its possible to also implement a promesa protocol to make other async things fulfill it, here is an example using core.async via said extension:
(extend-type cljs.core.async.impl.channels/ManyToManyChannel
  promesa.protocols/IPromiseFactory
  (-promise [this]
    (p/create
      (fn [resolve reject]
        (go
          (let [v (<! this)]
            (if (error? v)
              (reject v)
              (resolve v))))))))

(comment
  (let [start (system-time)]
    (-> (p.a.eql/process (pci/register
                           (pco/resolver 'foo {::pco/output [:foo-async]}
                             (fn [_ _] (go {:foo-async "the-value"}))))
          [:foo-async])

        (p/handle (fn [result error]
                    (println "RES" result error))))))
#2021-01-1616:16wilkerlucioI probably wont provide this in the library to avoid having the dep on core.async at all, but it can be a separated library, or a documentation page with the snippet#2021-01-1616:16wilkerluciowhat you think?#2021-01-1617:00mpenetGood approach. I really like core.async myself but having completablefutures is a good default, especially on the jvm. And it's easy to plug one into the others (you can make completablefuture impl for core. async readport/writeport that follows promise-chan impl, then overhead is really low I guess)#2021-01-1617:01mpenetI wouldn't bake in interceptor usage tho, this is more opinionated, there's no standard and there are multiple good impl. in clojure with subtle differences.#2021-01-1913:10dehliI think that’s a great solution! like you said it could be a separate library if there’s enough demand for it but it’s also a pretty small change that just mentioning it in the README should be enough#2021-01-1522:20nivekuilI like promesa too. I think it is really close to bare CompletableFuture overhead#2021-01-1522:21royalaidIs it possible to provide channel coercion in pathom3?#2021-01-1616:29wilkerlucioyes, Promesa has protocols the user can implement that can make coercion from other types: https://clojurians.slack.com/archives/C87NB2CFN/p1610813575011700?thread_ts=1610744517.006300&amp;cid=C87NB2CFN#2021-01-1522:21royalaidthen you could possibly get the best of both worlds#2021-01-1522:22royalaidcompatibility with core.async and performance of promesa#2021-01-1522:22royalaidit might also be worth looking at kitchen-async(https://github.com/mhuebert/kitchen-async)#2021-01-1616:29wilkerlucioyes, Promesa has protocols the user can implement that can make coercion from other types: https://clojurians.slack.com/archives/C87NB2CFN/p1610813575011700?thread_ts=1610744517.006300&amp;cid=C87NB2CFN#2021-01-1616:00Reily SiegelDoes Pathom3 provide the parser in the env? I want to be able to resolve attributes in mutations, and would presumably need access to the parser.#2021-01-1616:13wilkerlucioPathom 3 just uses the env directly for everything, no need for parser#2021-01-1616:14wilkerlucioyou can use the env to trigger p.eql/process from inside the mutation#2021-01-1616:14wilkerluciomakes sense?#2021-01-1616:14Reily SiegelYep, thanks#2021-01-1914:50wilkerlucioAsync supported landed on Pathom 3 master šŸŽ‰ documentation available at: https://pathom3.wsscode.com/docs/async/#2021-01-2005:13wilkerlucio#2021-01-2010:42thhellerhey, did you consider using a protocol for the async support so its not coupled to promesa or core.async or whatever and instead is extensible?#2021-01-2010:46thhellersomething like
(defprotocol IAsyncResult
  (on-result [this success-fn error-fn]))

;; conditional dispatch to async if needed
(if (satisfies? IAsyncResult result)
  (on-result result handle-success handle-error)
  (process-sequentially-somehow result))

;; in a js-promise ns
(extend-protocol IAsyncResult
  js/Promise
  (on-result [this success error]
    (-> this
        (.then success)
        (.catch error))))

;; in a promesa ns
;; same thing, only importing this ns is optional, API doesn't change
#2021-01-2010:47thhellersame for core.async and so on#2021-01-2014:05wilkerlucioPromesa has such protocols, so you can use them to extend other kinds of futures to conform to it, I did an example on how to make core-async compliant on the docs#2021-01-2014:06wilkerluciodo you have any concerns to extending from the promesa side?#2021-01-2014:17thhellerI don't like adding promesa as a dependency in my projects when I don't have to#2021-01-2014:20wilkerlucioI would have to add something, because of the primitives to run in pathom, I dont want to write a portable future impl between clj and cljs, that's why Promesa (which goes quite close to the core of ComputableFuture on the JVM and Promises on JS)#2021-01-2014:21wilkerlucioin contrast, no more dep on core.async#2021-01-2014:24thhelleryeah its probably fine. has been a while since I used promesa and it likely corrected some of the stuff they did in the past#2021-01-2014:25thhellerjust think its a generally good idea to not couple yourself to one specific async impl#2021-01-2014:26wilkerlucioyeah, I would totally prefer that, but the differences between environments are big enough that I think its worth in this case (also because Promesa is small, so in this case seems to do just what I need to it)#2021-01-2014:27wilkerlucioalso already offers the protocols for extensions, so if you want to use core.async, manifold or anything else on your resolvers/mutations, there is the extension point already#2021-01-2014:28wilkerluciowhat kind of issues you had with promesa in the past?#2021-01-2014:32thhellerit used to ship its own promise impl but appears to just use js/Promise now so thats good#2021-01-2014:32thhellerguess my main problem with it is gone then#2021-01-2016:15mpenetfor core.async there's a cheaper way than what you do in the example: you can just make completablefuture extend the writePort/readPort protocol I think, so core.async can take/put a cf directly, that saves creating a channel.#2021-01-2016:16mpenetI might just send a PR for this, I am juggling with too much stuff right now#2021-01-2016:19mpenetsomething like that https://gist.github.com/hiredman/1788aa052f26d127c00a1679656026f0 (not sure the impl on the gist is correct tho)#2021-01-2100:51wilkerluciobut by looking at your code I guess its doing the reverse of what I'm doing#2021-01-2100:51wilkerlucioin my case I'm making core.async channels compatible with promises, so I can use promesa and read from channels as-if they were promises#2021-01-2100:52wilkerluciowhile your example would allow a CompletionStage to be read as a channel#2021-01-2100:52wilkerlucio@mpenetĀ I started a library to have shared implementation of those, still using my approach for now, but happy to take pull requests for improvements:Ā https://github.com/wilkerlucio/promesa-bridges#2021-01-2015:57royalaidHey all, is there a recommended way to compose transforms? It looks like currently only one function is accepted so my first though is to just reach for comp#2021-01-2016:08wilkerlucioI was thinking about that yesterday#2021-01-2016:08wilkerluciothe idea I have is to make transport supports vectors, in the same way register does#2021-01-2016:09wilkerlucioso you could do as: ::pco/transform [t1 t2 t3]#2021-01-2016:10wilkerluciobut before that you can use comp, single param in and out, works fine#2021-01-2100:52wilkerlucio@mpenetĀ I started a library to have shared implementation of those, still using my approach for now, but happy to take pull requests for improvements:Ā https://github.com/wilkerlucio/promesa-bridges#2021-01-2116:14wilkerluciohello folks, short updates of the day, the tooling integration with Pathom 3 is going well, currently working with Pathom 3: - query editor* - index explorer - requests The query editor * is because there is no trace support for Pathom 3, the tracking of Pathom 3 is different and I'm still experimenting to see if I can reuse the same trace view or if it will need something new on top of previous features, these are new ones available: - graph view for Pathom 3 plans => this is currently showing up instead of the trace for Pathom 3, but this view is incomplete, because it only shows for the root plan, there are plans for every entity, this is were the new trace view needs to make a way to see everything - new logs tab => this is a new main tab on Pathom Viz, by using some new commands from pathom viz connector you can log visual entities there, currently only to render graph visualizations from plain structures (without having to connect an environment, here I did wrote a few Reveal extensions so I can transfer a map representation to a visual one to see in Pathom viz)#2021-01-2215:58royalaid@wilkerlucio is error handling for pathom3 still up in the air per the older blog post?#2021-01-2216:03wilkerlucioyeah, I'm having some ideas, but before I say them, I like to hear how you think a good error strategy would be#2021-01-2216:03wilkerlucioon Pathom 2 I copied what GraphQL was doing, and have just a separate branch of data for errors, and later add some helpers to move the error closer to the entity (for UI development that seemed a better strategy)#2021-01-2216:04wilkerluciowhat you think about these current ones? and do you have another idea about how they could be?#2021-01-2216:11wilkerluciocurrently depends on what you need. I think smart maps are fine, you can have errors out on them. for EQL there are more nuances. if you are not crossing the process, then each map contains all the data you need in the meta, you can use some helpers to find out about specific attribute errors. the wire is a different issue, during development is a good idea to send all the meta over the wire (this empowers pathom viz to do its things), but in prod its a lot of data to keep sending. my current idea is to have built-in plugins to handle that, they process the errors and include in the output, then we can remove the meta and just keep the errors#2021-01-2301:49royalaidI spent some time on this for a while today and think that the idea of "errors" is overloaded and so it hard to pin things down to a simple implementation. Your comments above highlight this, a user accessing something they aren't authorized to is very different than a NPE in a function that formats a string. Ultimately I couldn't really think of a way to add anything useful to what pathom 2 did.#2021-01-2218:05markaddlemanOn the subject of pathom errors. It would be convenient if the EQL response would also include the full exception in data (or object) form rather than string form. For example,
com.wsscode.pathom.core/errors: {
[[:data-source/id {:portfolioKey "xyz", :appKey "xyz"}] :entity.default]: "class com.google.cloud.bigquery.BigQueryException: 401 Unauthorized
POST "
[[:data-source/id {:portfolioKey "xyz", :appKey "xyz"}] :entities]: "class com.google.cloud.bigquery.BigQueryException: 401 Unauthorized
POST "
}
#2021-01-2218:05markaddlemanThis is from pathom2 ^^#2021-01-2218:13wilkerlucio@markaddleman thats a default that was picked because on that time Pathom was used mostly on the wire, but in highsigth was a bad pick. in pathom 3 the standard will keep errors as-is. you can change that in Pathom 2 adding the following to your env: ::p/process-error (fn [env e] e)#2021-01-2218:14markaddlemanah, that's great and makes my debugging life MUCH better#2021-01-2218:18wilkerlucioMVP of tooling for Pathom 3 is out šŸŽ‰ ! There is a new version of Pathom Viz app, you can download it at: https://github.com/wilkerlucio/pathom-viz/releases/tag/v2021.1.22-1. More important, there is an update to the connector library: https://github.com/wilkerlucio/pathom-viz-connector If you just update the connector library, you can use it with Pathom 3, updating the app gives you: - Support to see the root graph rendering (this view is a work in progress) - Logs tab (to log plans) - Improved auto-complete algorithm Here you can find an example usage (there is also in the README for the connector lib): https://github.com/wilkerlucio/pathom-viz-connector/blob/master/examples/com/wsscode/pathom_viz/connector/demos/pathom3.clj#2021-01-2218:18wilkerluciojust also make sure you are on pathom 3 latest commit#2021-01-2322:36wilkerlucioHello everyone! Sneak peak on the tracer for Pathom 3!! I was able to re-use the previous timeline, altough the format is a bit different, in Pathom 2 each row was an attribute, now they are tied to resolvers in the graph, getting some cool animations thanks to Cytoscape!#2021-01-2518:12kendall.buchananThis is phenomenal. In your opinion, how much closer does this push Pathom 3 to ā€œproduction-readyā€? Is this a big step, or small?#2021-01-2519:43wilkerluciofeature-wise is close to complete, the tooling should be out this week, so for MVC I say its a small step. production-ready to me also means the library is in use in production and really tested in real applications. currently I'm developing Pathom 3 without any such app to test it for good, so I will need have people trying it out. since the new planner algorithm is new and quite complex, as the complexity of users graphs grow, I expect early users to find bugs on it (when I was porting repl-tooling I found a few, but I guess there is more out there). but once a few ups are up and running good, than I say its prod-ready, but also depends on people using it to get there šŸ˜‰#2021-01-2520:38kendall.buchananšŸ‘ Gotcha.#2021-01-2509:48imreHey Wilker, I was wondering if you could point me in the right direction with this.
{:foo/people
 [:foo.person/name
  :foo.person/age
  :foo.person/family-tree]}
Imagine the above. I am trying to wrap a library call which is roughly (input, calculate-family-tree?) -> people. Now, family tree calculation is expensive, so when it is not needed, I'd like to pass false there. Is this solvable only by looking at the query itself in the resolver, or is it possible to make this work by writing multiple resolvers with different output definitions, one resolver also returning family trees, the other not. The aim is that there is only one call made to the lib and calculate-family-tree? is false where we don't need to return it.
#2021-01-2513:37dehliIf you have a resolver that only outputs :foo.person/family-tree and then another resolver for the other attributes I think that would solve the issue. If the client doesn’t specify family-tree in the query, then pathom won’t execute the expensive query. Does this answer the question?#2021-01-2513:42imreI'm not sure that would work. Clients either want [name age family-tree] or [name age].#2021-01-2513:43imreAnd while family-tree is the most expensive to calculate, the other 2 are also somewhat costly, so we only want to make one call to the underlying lib. Retrieving family-tree separately from the other 2 won't work for us I'm afraid.#2021-01-2514:36imreOne alternative we considered was to make 2 top-level keys, :foo/people and :foo/people-with-family-tree, which works but feels a little clunky.#2021-01-2519:24wilkerluciohello @U08BJGV6E, not sure if I got it right, but the thing you say is that to compute :foo/people you like to know ahead of time if the user asked for family-tree in the sub?#2021-01-2519:25wilkerluciomy normal reaction to this would be to have one resolver for each attribute there (name, age, family tree)#2021-01-2519:25wilkerluciothe only case I think you really need to see it before is that if your sub request depends on it, in this case its more like Datomic or GraphQL integrations (where we need to figure what parts of the sub-query must be delegated), does this sounds like your case?#2021-01-2519:33imreOne resolver per attr would be best, yes, but the library we are wrapping returns all 3 of these in one call getpeople(input, family-tree?) -> people#2021-01-2519:34imreand the call is expensive, and even more expensive if we tell it to also return the family tree#2021-01-2519:35imreso regardless of the desired output (whether it's a single prop or all 3) we only want to make one lib call per request#2021-01-2519:36imreand when the request doesn't specify family-tree as output, pass false to the lib call so it isn't calculated#2021-01-2519:37wilkerluciogotcha, that to me aligns a lot on how dynamic resolvers work, this is something I'll work more once the tools and basics are out#2021-01-2519:38wilkerluciofor now, like you said, you can look at the ast and make this decision#2021-01-2519:38wilkerlucioI think I did something similar for the Youtube API, where they accept the "parts" to say which pieces you want, in this case I also look at the query to decide it#2021-01-2519:38wilkerluciowhat's really tricky is if you don't depend on this attribute directly, but instead you are trying to get some attribute that depends on it#2021-01-2519:38wilkerlucioon this case, the simple lookup on the AST wouldn't be enough#2021-01-2519:39wilkerluciothat's where dynamic resolvers will be really helpful, because they can do these attribute calculations for you (and it does it at the same planning phase as the static resolvers stuff)#2021-01-2519:40imreGotcha, thank you. What do you think about the approach where we'd have 2 resolvers, one returning {:foo/people [:name :age]} and another returning {:foo/people-with-tree [:name :age :tree]}?#2021-01-2519:42imreI wonder if you'd consider this more or less fragile in terms of composition than the look-at-ast way?#2021-01-2519:48wilkerlucioare you in pathom 2 or 3?#2021-01-2519:48imre2#2021-01-2519:48wilkerlucioI think you can do with two resolvers, but you would need pathom 3, to set the resolver priority#2021-01-2519:49wilkerlucioalso, pathom 2 doesnt look at subqueries for process, pathom 3 does#2021-01-2519:49imreI see. Thank you for the insight!#2021-01-2519:49wilkerlucioso {:foo/people [:name :age]} needs to have higher priority if you do it this way#2021-01-2519:49wilkerlucioah, sorry#2021-01-2519:50wilkerlucioif you use different names foo/people and foo/people-with-tree, that works fine (in pathom 2 or 3)#2021-01-2519:50wilkerlucioif you don't mind having this caveat for your interface, its good#2021-01-2519:52JanosHey Wilker, I've been trying to use this helper fn for the dynamic resolver for this problem: https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/connect.cljc#L670#2021-01-2519:54JanosUnfortunately we would need nested attributes in the response like in your latest example {:foo/people [:name :age]}#2021-01-2519:55JanosFor this query that fn would only return :foo/people attribute#2021-01-2519:59JanosBut I guess I could write some custom code 😊#2021-01-2520:12wilkerlucioyeah, if you need, go forward, Pathom 2 isn't getting any updates on the dynamic side of things, and for Pathom 3 things are quite different, so whatever happens in Pathom 3 is gonna be different from what you looking at now#2021-01-2521:00wilkerluciohello people! for the adventurous, the new Pathom Viz with trace support for Pathom 3 is outĀ šŸŽ‰! You can download it at:Ā https://github.com/wilkerlucio/pathom-viz/releases/tag/v2021.1.25, remember to use the latest connector and pathom 3 as well. Documentation around debugging (with and without Pathom Viz) is my next task, should be out later this week! Even if you don't use Pathom 3 yet, I recommend upgrading, I updated the views with Clojure readonly to use the new Clojure-mode from nextjournal, and its great! the biggest difference for me is that it supports folding, and when looking at big structures that's a big deal. Also I made all the printed maps to be sorted, making easier to find the data. Remember Pathom Viz is compatible with both Pathom 2 and 3. Please let me know if have any issues with this release.#2021-01-2620:12frankitoxHi, I'm trying to use ::p/process-error to print dummy string to console if pathom throws while resolving the query. It works if the query uses a resolver, but it's not printing if the query is a mutation. Is that the default behavior?#2021-01-2621:07wilkerlucioit should work with mutations, can you send a reproducing example of what you see?#2021-01-2622:31frankitoxAhh, I see. I'll let you know if I can reproduce it, but most likely I screw it somewhere šŸ˜…. Thank youu!#2021-01-2621:08wilkerlucioI just released documentation for how to debug in Pathom 3 šŸŽ‰ ! This covers how to examine process details via REPL and using the Pathom Viz application, check it out at: https://pathom3.wsscode.com/docs/debugging#2021-01-2621:39wilkerlucioAlso published a new release of Pathom Viz, this fixes a bug with recent commits from Pathom 3, also fixes re-run of queries (trace was stuck in this case). And to finish it now has a link for setting up with Pathom 3 at the initial screen, download at: https://github.com/wilkerlucio/pathom-viz/releases/tag/v2021.1.26-1#2021-01-2623:04alex-eberts@wilkerlucio Awesome job on the documentation for Pathom! It’s one thing to write a useful library but if the docs aren’t good chances are that no one will use it… The Pathom docs are clear without hiding any details and the examples are fantastic. Kudos!! šŸ‘#2021-01-2705:28royalaidHey all, trying to build pathom-viz on windows here and it looks like in order to get shadow to actually compile I had to run:
npx shadow-cljs --aliases dev:dev-ws:electron:provided release electron-background electron-renderer
this replaces the
./script/build-electron
step in the docs
#2021-01-2714:05wilkerluciodid you tried on master? I tough I fixed that yesterday#2021-01-2714:07wilkerluciobut I also didnt got to test on windows, after that command it ran fine for you?#2021-01-2714:15royalaidYeah, I have pathom viz running#2021-01-2714:15royalaidIt seems like the exe isn't one file and you have to keep it in the build output location#2021-01-2714:15royalaidbut other than that, it seems to work#2021-01-2714:37wilkerlucioglad to hear, please let me know if you find any issues#2021-01-2714:39royalaidcopying from the query output seems... odd? Like I think I can paste into that field#2021-01-2714:39royalaidand using node+async seems to cause issues with the autocomplete, all my resolvers don't show up in autocomplete, and the trace seems incomplete an ungraphed#2021-01-2714:40royalaidAnd those resolvers do show up in indexes#2021-01-2714:40royalaidI am not sure its my graph being poorly built or if it is something with pathom viz#2021-01-2714:41royalaidAlso if I toss a query that is malformed or impossible I lock up the UI#2021-01-2714:45wilkerlucioresolvers don't show auto-complete, just the attributes#2021-01-2714:45wilkerlucioasync should be ok, Pathom 2 or 3?#2021-01-2714:46wilkerlucio(also, attributes are contextual, the options depend on your query path)#2021-01-2714:57royalaidpathom3#2021-01-2714:58royalaidand to be fair, I am just assuming async is the problem which, on reflection, probably doesn't make a lot of sense#2021-01-2715:02royalaidre the resolver and attributes comment, I meant the attributes that I have added via resolvers#2021-01-2715:02royalaidpathom's built-in's seem to show up#2021-01-2715:41wilkerlucioprobably a dependency thing#2021-01-2715:41wilkerlucioat root you only see what is globaly available#2021-01-2715:41wilkerluciobut if you start a query, let's say: [{[:user/id 123] [CURSOR_HERE]}]#2021-01-2715:41wilkerlucioat that point, it should show up things that are reachable from a :user/id#2021-01-2715:43wilkerluciosupport for data from Placeholders is coming soon#2021-01-2715:46royalaidAhh problem seems to be that I rely on two inputs to my resolver#2021-01-2715:57wilkerlucioif you have some entry point that provides both, them you can see auto-complete for it#2021-01-2716:30msshey everyone, what’s the preferred way to inject query data that’s already been resolved in a parent/sibling key into a child query input? attempting to put the relevant key on my input declaration doesn’t seem to be working#2021-01-2716:36mssI figured it would be possible to do something like ::pathom-connect/input #{:user/email :other-thing/id} but that doesn’t seem to be the case#2021-01-2716:41wilkerlucio@mss if I get right, what you need is to place the data in the output of the parent, so it will be available for children#2021-01-2716:41wilkerluciothe input is a declaration of requirement, the data needs to come from somewhere, can you give a small example of what you are trying to do?#2021-01-2716:45mssyeah so I have a fulcro load! call on app startup that’s parameterized by :user/email like so:
(df/load! mounted-app
 [:user/email user-email]
 User...)
that load! hits a series of resolvers that pull user app state. a user has projects, which have goals, which have tasks. within the the resolver for a task, I want to be able to utilize that :user/email key that’s theoretically already resolved by the input entity to pull some state within the resolver. for whatever reason, declaring a resolver with input of #{:user/email :task/id} doesn’t seem to be getting triggered. I’m sure I’m missing something simple here, just not sure what
#2021-01-2717:05wilkerluciothis a good time to debug just your parser, use the REPL and make calls direclty to it, use * on the query to see all the data available at that point#2021-01-2717:05wilkerluciothis way I usually can spot where the data is missing#2021-01-2718:04souenzzo@mss in pathom3, placeholders do that https://pathom3.wsscode.com/docs/placeholders in pathom2, you can use :pathom/context special params Your query should look like that:
[{([:user/email "abc"] {:pathom/context {:other-thing/id 42}})
  [:user/something]}]
#2021-01-2720:23mss@wilkerlucio what specifically would I be looking for? my query is calling the correct resolver unless I put in the key :user/email in the input set like so:
{::pathom-connect/input  #{:user/email :task/id}
   ::pathom-connect/output [:task/title
                            :task/description]}
the parent/sibling resolvers are getting correctly invoked given an input set of #{:user/email}. not quite sure how to debug that at the repl
#2021-01-2720:28mssI would think that since :user/email is both a key satisfied by a query parameter/ident and returned into the response map at my top level user resolver that it would be available as an input key to child resolvers#2021-01-2720:30mssFWIW my query looks something like this:
[{[:user/email "
and my top level user resolver looks roughly like this:
(pathom-connect/defresolver user [env params]
  {::pathom-connect/input  #{:user/email}
   ::pathom-connect/output [:user/first-name
                            :user/last-name
                            :user/email]}
  {:user/first-name user-first-name
   :user/last-name  user-last-name
   :user/email (:user/email params)})
#2021-01-2720:36mssI would have expected that :user/email would then be available in child resolvers#2021-01-2721:42wilkerluciothe ident only provides data to the first child level of it, after that each context has only the data provided for it#2021-01-2721:43wilkerlucioyou can flow the data down manually if you want, but seems too many levels for me#2021-01-2721:43wilkerlucioanother option is have a path to fetch the user id from the task id (if it has a reference to one)#2021-01-2722:31mssWhat would a path to fetch the user id look like? Something out of band? My initial way of solving this was to reach for storing global data like that in the request and throwing it into the parse env. Is that the best way to go about that?#2021-01-2722:35mssyeah unfortunately within the task resolver there’s no context around what user id exists. I tried to assoc the user/email into the env within the user resolver, but that didn’t seem to work#2021-01-2722:40wilkerlucioit depends on how you data is organized, does your db row/entity for task knows about the user related to it?#2021-01-2722:40wilkerluciodoes it have a reference to it?#2021-01-2722:40wilkerlucioa path is just a resolver that gets :task/id and outputs :user/id, pulling that relationship from your database#2021-01-2722:41wilkerluciobut this also depends if your data model a task has a single ID, or if its something else#2021-01-2723:17mssThe task knows nothing about the user, unfortunately. It’s currently designed as a one way relationship from users to projects to goals to tasks#2021-01-2723:26wilkerlucioin this case I suggest you do the manual forward down of the user id#2021-01-2723:26wilkerluciofor example, in the reoslver for :user/projects, in each item you add the :user/id you got on that resolver input#2021-01-2723:27wilkerluciothen you do the same for :project/goals, then same for :goal/tasks, makes sense?#2021-01-2723:53mssYep, absolutely. Really appreciate the insight here. And thank you for all the work you’ve done w pathom — it’s phenomenal software#2021-01-2717:07jmayaalv@wilkerlucio having several problem with the new pathom-viz app, it’s constantly locking. couldnt’ really see anytyhing in the console log. how can i debug it? i noticed that everytime i paste something app blocks, this is not the only case though.#2021-01-2717:07wilkerlucioyou can open the devtools with cmd + opt + i (on mac). or ctrl + alt +i on others#2021-01-2717:07wilkerluciowhat OS are you in?#2021-01-2717:08wilkerlucioand you mean pasting at the query editor?#2021-01-2717:08jmayaalvmac (bigsur), dont’ really see anything on the devtool, it just freezes when writing the query#2021-01-2717:11jmayaalvyes, pasting at the query editor. but it’s freezing also when comlpeting a query with an input. example, i am trying to type this
[{[:edge.calendar/code "{{code}}"] [:edge.calendar/name :edge.calendar/code   :edge.calendar/calendarid  :edge.calendar/creation-date :edge.calendar/expired
                                                                    :edge.calendar/status :edge.calendar/workdays  :edge.calendar/weekend
                                                                    {:edge.calendar/events [:edge.calendar.event/day :edge.calendar.event/type :edge.calendar.event/label]}]}]
so when i get here it freezes.
[{[:edge.calendar/code "acode"] []}] 
#2021-01-2717:11jmayaalvthis is against a pathom2 engine#2021-01-2717:16jmayaalvsaw this is in the console when connecting to the remote
main.js:2441 POST  500 (Internal Server Error)
h	@	main.js:2441
l	@	main.js:2442
$APP.cljs.core.async.impl.ioc_helpers.run_state_machine	@	shared.js:7233
$APP.cljs.core.async.impl.ioc_helpers.run_state_machine_wrapped	@	shared.js:7234
(anonymous)	@	main.js:2443
$APP.cljs.core.async.impl.dispatch.process_messages	@	shared.js:7166
processImmediate	@	internal/timers.js:439
#2021-01-2717:17jmayaalvno error when the app freezes, just when connecting to the remote#2021-01-2717:19wilkerluciothe engine shouldn't matter because the calculations happen inside the app for auto-complete#2021-01-2717:19wilkerlucioI didn't upgraded to Bigsur, but I heard about some general complaints on performance, I wonder if the version of Electron (which is not very recent) is getting affected by it#2021-01-2717:20jmayaalvmight be, i didn’t use pathomviz since upgrading to big sure, let me check if the older version of PathomViz has the same issue#2021-01-2717:21wilkerluciocool, thanks for checking, curious to learn of your findings#2021-01-2717:22jmayaalvno problem with 1.1.1#2021-01-2717:22jmayaalvpasting works, query i was trying to enter works fine#2021-01-2717:24jmayaalvso i think something it’s going on with the latest version. i am happy to help test it, not sure how though :/#2021-01-2717:26wilkerluciois it a big index? I did changed the auto-complete algorithm, but I personally only tested on small things, it can it#2021-01-2717:27jmayaalvno, it’s not big at all#2021-01-2717:29wilkerlucioif you have some time to help with debug, you can build the app locally, and try there, using the performance tools from Chrome, in the dev app you will be able to see the names of the functions#2021-01-2717:29wilkerluciohave you used the performance tab from Chrome before?#2021-01-2717:30wilkerlucio(and just to give you context, even if the algorithm was slower, there is a bunch of caches, so it should be slow only for the first letter, after that its the same, so I wonder if the cache is misconfigured)#2021-01-2717:30wilkerlucioor maybe its something else entirely#2021-01-2717:33jmayaalvcould it be the cache from the prev version interfering with the latest? ill remove it and try again#2021-01-2717:35wilkerluciono, this cache is memory only#2021-01-2717:36jmayaalvok, will check the performance tab and send you my findings.#2021-01-2717:44jmayaalvthis a screenscast with the issue#2021-01-2717:46wilkerlucioyeah, and the view is strange too, it shouldn't be black on the right side#2021-01-2717:46wilkerluciohow is the style after you run a query?#2021-01-2717:46jmayaalvthe black is the console#2021-01-2717:47wilkerluciobefore that, the query results pane part#2021-01-2717:47jmayaalv(dark theme)#2021-01-2717:47jmayaalvohh right#2021-01-2717:47wilkerluciofor comparison here#2021-01-2717:47wilkerluciodark theme shouldn't affect it (unless there is something I dont know about Codemirror6 and Bigsur :P)#2021-01-2813:36nivekuilthe plugins are so much nicer to write in pathom 3!#2021-01-2814:23wilkerluciothanks šŸ™‚ what change you found made it nicer in your opinion?#2021-01-2814:31nivekuilit's much shorter, for one.
(def mulog-trace-plugin -  {::pc/wrap-resolve -   (fn [resolver] -     (fn [env input] -       (u/trace (-> env ::pc/resolver-data ::pc/sym keyword) -         [] -         (resolver env input)))) -   ::p/wrap-mutate -   (fn [mutate] -     (fn [env k params] -       (let [out (mutate env k params)] -         (cond-> out -           (:action out) -           (update :action -                   (fn [action] -                     (fn [] -                       (u/trace (keyword k) -                         [] -                         (action)))))))))})
vs
(defplugin mulog-trace-plugin   {::pcr/wrap-resolve    (fn [resolver]      (fn [env node]        (u/trace (-> node ::pco/op-name str) []                 (resolver env node))))    ::pcr/wrap-mutate    (fn [mutation]      (fn [env ast]        (u/trace (-> ast :key str) []                 (mutation env ast))))})
#2021-01-2814:33nivekuilit generally just feels much smoother to work with operations directly, as you can call them like functions and mutations/resolvers seem to share more structure than in p2#2021-01-2814:35nivekuilmore fun to work with means I actually write plugins. for example, I think pc/transform is not in p3 but it was really easy to just write a custom plugin to replace it#2021-01-2814:43nivekuilin general I think a lot of small changes added up to make p3 feel much more streamlined#2021-01-2817:10wilkerluciothank you very much for the feedback šŸ™#2021-01-2817:11wilkerlucioyeah, mutation plugins were a real pain in the ass in Pathom 2#2021-01-2817:11wilkerluciojust one thing to note though, what you get in the plugin isn't the resolver, neighter the mutation#2021-01-2817:11wilkerlucioits the functions that wrap their call inside the runner#2021-01-2817:12wilkerlucioif you want to access the actual resolver or mutation, you must use the node on resolver, and ast + index in the mutation#2021-01-2817:44nivekuilyeah, I figured out something like this: (get-in env [::pci/index-mutations (:key ast) :config]) not sure if it's mentioned in the docs#2021-01-2817:45wilkerluciono, if you like to add PR is welcome, or I can put in a todo list here#2021-01-2908:36nivekuilprobably should be you, I don't know enough about best practices :)#2021-01-2900:37markaddlemanI pathom3 error handling is still undergoing active thought so I'll just through this out there: Using ::psm/error-mode-loud and guardrails enabled, if the smart map cannot compute the requested key, guardrails throws an unhelpful error. When guardrails is disabled, the smart map returns nil. I'd like to see a useful error message under the loud case regardless of whether guardrails is enabled or not.#2021-01-2900:38wilkerlucioin loud mode it should throw an exception in case something goes wrong, independent of guardrails#2021-01-2900:38wilkerlucioif you are seeing something different, is probably a bug, can you send a reproduction case?#2021-01-2900:39wilkerluciofor EQL mode there is a new plugin that I'm about to document, that exposes the errors on the data#2021-01-2900:40wilkerluciohttps://github.com/wilkerlucio/pathom3/blob/master/src/main/com/wsscode/pathom3/connect/built_in/plugins.cljc#L10#2021-01-2902:13markaddlemanIt turns out this is a me problem and not a pathom problem. The smart map is working exactly as you describe#2021-01-2923:57markaddlemanWell, I'm not longer sure this is a me problem šŸ™‚#2021-01-2923:59markaddlemanWhen guardrails is enabled, accessing :a/output from m results in an unhelpful guardrails exception.#2021-01-3000:01markaddlemanwhen a->b's input is available, the result is what you'd expect#2021-01-3000:08markaddlemanwhen guardrails is disabled and a->b's input is not available, the result is nil . I'm agnostic as to whether this is desired behavior or not.#2021-01-3013:40wilkerluciojust checked on it, that guardrails messages are not wanted, something is wrong inside, checking now#2021-01-3014:21wilkerlucio@U2845S9KL should be fixed on master, by this commit: https://github.com/wilkerlucio/pathom3/commit/a6d9c3430f36a3b48a0952bd812fbfb3eb4dc37a#2021-01-3014:26wilkerlucionow there is a third type of error ::node-error-type-unreachable, which covers the case where Pathom can't find a path for some attribute#2021-01-3016:22markaddlemanThanks!#2021-01-2901:07wilkerlucioNew page for Pathom 3 docs: built-in plugins šŸŽ‰ ! https://pathom3.wsscode.com/docs/built-in-plugins#2021-01-2908:36nivekuilit seems like attribute-errors-plugin causes a massive slowdown, from ￱￱55ms to ￱￱200ms on my api call#2021-01-2913:52wilkerluciogonna check on that, how big are your queries?#2021-01-2914:00wilkerlucio@U797MAJ8M can you try again on master please? I made a change that will remove the checks when no errors happen, this should make the standard path (when no errors happen) to stay the same speed#2021-01-2914:06nivekuil@wilkerlucio: much better now, no apparent perf impact in repl#2021-01-2914:07wilkerluciocool, being honest, you will probably see a bit more when some error happen, but at least will be only for the entries that have error#2021-01-2914:07wilkerluciothis cost comes from plan traversing to find the errors, the plugin will do it once for each attribute you requested, and in case of a no-error it has to traverse back until the root, so the cost will highly depend on the graph complexity#2021-01-2914:08wilkerlucioI think this can be optimized in the future to reduce the number of required lookups#2021-01-2903:46royalaidIs there a recommended way to test pathom resolvers? I would prefer to not have to call my create function before each test of a resolver#2021-01-2913:36dehlišŸ‘‹ in pathom3 you can invoke your resolver as if it were a function. not sure if that works in pathom2 as well#2021-01-2907:16jmayaalv@royalaid if you are in pathom2 you can call the resolver function from your tests like this:
((::pc/resolve your-resolver) context params)
in pathom3 you can call the resolver directly https://pathom3.wsscode.com/docs/tutorial#resolvers
#2021-01-2907:17jmayaalvnot sure if that’s recommended way for pathom2 though šŸ™‚ that’s how we do it.#2021-01-2909:04nivekuilcan map data be combined with EQL idents https://pathom3.wsscode.com/docs/eql#providing-map-data? I am trying to provide both in an eql query but it seems pathom is ignoring at least one of them#2021-01-2913:22wilkerlucioexample?#2021-01-2909:13nivekuilI ended up just putting the initial attributes into env and proxing them to params through a resolver#2021-01-2912:39souenzzo@wilkerlucio Will pathom3 implement parameters like :xform, :as, :default or it will be done only by external plugins?#2021-01-2913:22wilkerlucionot planned to do any of that, make your plugins#2021-01-2913:47dehlicould the notion of ::pc/mutation-join-global also be done by an external plugin?#2021-01-2913:53wilkerluciofor that there is a new key, the new key is called :com.wsscode.pathom3.format.eql/map-select-include, the attribute-errors-plugin does it: https://github.com/wilkerlucio/pathom3/blob/master/src/main/com/wsscode/pathom3/connect/built_in/plugins.cljc#L16-L22#2021-01-2913:54wilkerlucioah, sorry, for mutations is different, but doable as well, if you make a mutation plugin you can do the same there#2021-01-2914:17dehligreat! i will investigate. would there be interest in having a built-in plugin to support that old behavior? if i get a solution I can submit a PR if you’d like#2021-01-2914:18wilkerlucioonly if the feature makes sense as new feature for Pathom 3, but not for compatibility. its on my plans to have yet another library, to make the porting easier, any things that are related mainly for porting should go there#2021-01-3110:24Tomas BrejlaHello Wilker (and the community here!). I'm trying to learn pathom (version 2 for now; looking forward to release of version 3) and have following question, a bit related to the project-query-attributes function mentioned a few days ago by Janos in this thread https://clojurians.slack.com/archives/C87NB2CFN/p1611604378029100?thread_ts=1611568102.021800&amp;cid=C87NB2CFN project-query-attribute internally uses ::p/parent-query from env. I also found out that it's possible to access ::p/parent-query and use it to try to figure out, which of the outputs (fields) are actually being requested in that specific runtime situation. I wonder whether following approach is a good idea or it will hurt the query planner and explode in my face later. Let's say I have a resolver capable of returning :product/name, :product/price, :product/description etc. Let's say there's 20 such fields. Internally, the initial "naive" implementation executes SQL query that fetches all the 20 columns from the database table and returns them to pathom for further handling. This works fine, but it's obviously not very effective to always fetch all the columns' values when the client only asked for let's say :product/name and nothing more. In some of my simpler resolvers, I was able to peek into ::p/parent-query and make the SQL query dynamic and thus more efficient by only querying fir those columns that are actually required. But I'm not sure if it's a good idea or not. I have a feeling that it could break some internal functionality such as the filling the dots you often mention in your talks. What's the contract here? Should the resolver always return all the defined outputs, or is it ok for it to be a bit more clever and select just those fields which are requested by the actual runtime query? Sorry if it's explained somewhere in the docs and I just missed it.#2021-01-3123:33wilkerluciohello Tomas, looking at ::p/parent-query is usually not enough, the problems is that maybe you need something that's not on query, but is a dependency that will show up at planning. this is what project-query-attribute tries to figure out. You can play around, but it works different in Pathom 3, Pathom 3 planner does way more, and also knows about dynamic resolvers (not really right now, I did some experiments, but this area needs more work, and I should start on this soon).#2021-01-3123:33wilkerluciothe way Pathom 2 does is quite inefficient, because each resolver that needs to know about this hidden dependencies has to do its own work#2021-01-3123:33wilkerlucioin Pathom 3, this is all done in a single pass during the planning phase#2021-01-3123:34wilkerlucioI hope to provide a friendly user API to create these kinds of resolvers#2021-02-0100:02alex-ebertsHi all - quick question about EQL syntax. How does Pathom decide if a transaction refers to a mutation or a parameterized query (as far as I can tell, the syntax is the same). It is just up to the developer to make sure that mutations do not accidentally have the same names as resolver inputs?#2021-02-0100:55wilkerluciomutations are always symbols#2021-02-0100:59alex-ebertsAh, of course. facepalm Thanks!#2021-02-0104:06Reily SiegelWill it be possible in pathom3 to link a remote pathom resolver? IE, system1 has a set of resolvers/mutations, and system2 has a set or resolvers/mutations and the ability to remotely call resolvers/mutations from system1?#2021-02-0105:04wilkerlucioyes, this is part of the dynamic resolvers feature in pathom 3#2021-02-0113:47Reily SiegelIs there an approximate timeline for.this feature, or is it still being planned?#2021-02-0113:59wilkerluciocant promise times, Im finishing some docs adjusts, dynamic resolvers is the thing I want to work next, my guess is to have something in about 1 ~ 2 weeks#2021-02-0114:00Reily SiegelThanks for the rough estimate! Sorry to sound a bit like a nag, but I am trying to use pathom3 in a (non-commercial) project, and it has been great to work with so far! Thanks for all the work you put in!#2021-02-0113:41jjttjjIs pathom appropriate for situations where you have a graph of interdependent calculations? So not just fetching a value from a DB, but calculating a value of an entity, possibly based on other calculated values? So in other words, could it be used as a data flow mechanism? I've been starting to read the docs and play around but haven't been able to determine yet if this is a good use case#2021-02-0113:57wilkerlucioyes, this is the main purpose of pathom šŸ‘#2021-02-0114:15nivekuilI know ::pco/batch in p3 isn't documented yet, but I was trying it anyway and see some weird behavior where the batch resolver params contains duplicate values. Is this expected?#2021-02-0115:21wilkerluciohttps://pathom3.wsscode.com/docs/resolvers#batch-resolvers
#2021-02-0115:26nivekuilhuh.. bit concerned as to how I missed that. thanks for sharing the link :)#2021-02-0115:28wilkerluciono worries#2021-02-0115:28wilkerlucioat the top right there is also a search box#2021-02-0116:27nivekuil
(defresolver foo [input]   {::pco/input  [:foo/id]    ::pco/output [:foo/prop {:foo/bars [:bar/id]}]    ::pco/batch? true}   [{:foo/prop "foo"     :foo/bars [{:bar/id 10}                {:bar/id 20}]}])  (defresolver bar [input]   {::pco/input  [:bar/id]    ::pco/output [:bar/prop {:bar/foo [:foo/id]}]    ::pco/batch? true}   (mapv (fn [v] {:bar/prop (str "bar-" v)                :bar/foo  {:foo/id 3}})         input))  (def test-env (pci/register [foo bar])) (p.eql/process test-env [{[:foo/id 1]                           [{:foo/bars [:bar/id :bar/prop {:bar/foo [:foo/id]}]}]}]) 
#2021-02-0116:27nivekuilso if I run this query:
(process-query [{[:foo/id 1] [{:foo/bars [:bar/id :bar/prop {:bar/foo [:foo/id]}]}]}])
I get this result:
{[:foo/id 1] {:foo/bars [{:bar/id 10,                           :bar/prop "bar-{:bar/id 10}",                           :bar/foo {:foo/id 3}}                          {:bar/id 20,                           :bar/prop "bar-{:bar/id 20}",                           :bar/foo {:foo/id 3}}]}}
looks good
#2021-02-0116:28nivekuilIf I add one prop to the join:
(process-query [{[:foo/id 1] [{:foo/bars [:bar/id :bar/prop {:bar/foo [:foo/id :foo/prop]}]}]}])
I get Batch results must be a sequence and have the same length as the inputs.
#2021-02-0116:28nivekuilwhy is that? it doesn't seem like the cardinality has changed?#2021-02-0116:44wilkerlucioyour batch foo resolver is always returning a single item, by quick looking I think thats the problem#2021-02-0116:45wilkerlucioit needs to take the input size in consideration, and always return a list of same size as the input#2021-02-0116:46nivekuiloh, yes that's right. a bad example then sorry#2021-02-0118:20Reily SiegelI'm having an odd issue with pathom-viz it appears that when using the latest version, the application freezes when attempting to enter a join in the query editor. This happens as soon as the second vector (the one inside the map) is entered. This occurs whether the query is typed or pasted into the field. Responsiveness of the application can be restored by reloading it, but this does not fix the issue. While the app is unresponsive, the CPU usage also jumps considerably.#2021-02-0119:43wilkerluciowhat OS are you in? using pathom 2 or 3?#2021-02-0119:44Reily SiegelArch linux, pathom3. I tested both the Appimage and the Deb package#2021-02-0119:46Tomas BrejlaI managed to experience the same behavior you describe few days ago. I dumped my uncommited local git changes and found out it worked just fine again. So the problem must have been somewhere in my resolvers. I'll try to look if I still have that broken version in stash somewhere.#2021-02-0119:47Tomas Brejla(ubuntu budgie, deb package)#2021-02-0119:51wilkerlucioin any case, no user code should freeze the app, I'm interested in figuring this one out, just don't have the time right now, but would be glad to try it later#2021-02-0119:51wilkerlucio@U6NJBB596 is this some code you can share? I would like to try reproducing on my end#2021-02-0119:52wilkerlucioor if you can't, a minimal reproduction case would be great#2021-02-0119:53Reily SiegelThe code I'm using is in ReilySiegel/pod on GitHub, but it's not really in a great state at the moment (ie, no documentation, repl only, and the code structure is still changing). If you want to give it a try, init/halt functions are in com.reilysiegel.pod.server#2021-02-0119:54Reily SiegelI will work on a minimal reproduction as soon as possible, as that will likely be much easier for you to work with.#2021-02-0121:13Tomas BrejlaI believe I managed to find the minimal reproduction case of the freezing that happens in my case. When I define an ident-resolver that expects :product/id value on input and I also include :product/id value in ::pc/output, Pathom Viz freezes when I either copy-paste the query into its editor or when I type it manually. In that case it freezes in the moment when I type the opening bracket of the vector after ident (normally the code-completion popup would show). I have commented the :product/id with #_ in the example snippet. With this change, Pathom Viz doesn't freeze. If I uncomment it, restart Pathom Viz and re-connect by re-evaluating this whole namespace, Pathom Viz starts freezing again.#2021-02-0121:34wilkerlucioawesome, that seems an easy one to fix, I'll take a look at it later, thanks for the report!#2021-02-0121:35Reily SiegelThanks for beating me to a minimal reproduction!#2021-02-0121:39Tomas BrejlaSorry Reily if you wanted to to figure that yourself and I spoiled that to you šŸ˜‰. It just happened that the exactly same issue as you described happened to me a few days ago and I wasn't sure why. Luckily I git-stashed that weirdly-behaving code, so that I can return to it later for further inspection of the issue.#2021-02-0123:48wilkerluciofound the problem in Pathom, fixed on master, also generated a new build for Pathom Viz: https://github.com/wilkerlucio/pathom-viz/tree/v2021.2.1#2021-02-0123:48wilkerluciocan you confirm this build works without freezing?#2021-02-0200:19Tomas BrejlaSeems to have fixed the issue for me :thumbsup: šŸ‘ Thanks!#2021-02-0209:05jmayaalvNice !!! i was having the same issue and couldn’t really find a reason! The new build solves the problem !!!#2021-02-0209:07jmayaalvThank you !!!#2021-02-0209:23wilkerlucioPathom Viz 2021.02.01 is out! This fixes some freezing bugs on auto-complete, if you are on recent versions please be sure to update! https://github.com/wilkerlucio/pathom-viz/tree/v2021.2.1#2021-02-0316:48souenzzoIn pathom3, there is some function that given a an env and a query it returns which inputs are required to resolve this function? for example: - env: (pci/register [(single-attr-resolver :a :b inc)]) - query: [:b] - returns: [:a]#2021-02-0317:09wilkerluciothat's the planner#2021-02-0317:13wilkerlucioit wont tell you directly that, but you can create a plan and infer this data from it#2021-02-0317:24jmayaalv@wilkerlucio i am testing the batch resolvers on pathom3 and found a problem. If the ā€œparentā€ resolver of the batch resolver returns a seq you get a 1. Unhandled java.lang.ClassCastException this is a way to reproduce it:
(def db {:contracts {"contract-1" {::contract/id   1
                               ::contract/code "contract-1"
                               ::contract/name "the name"}}
         :positions {1 '({::position/units  10
                         ::instrument/fund {::fund/id 1}}
                        {::position/units  20
                         ::instrument/fund {::fund/id 2}})}
         :funds     {1 {::fund/code "fund 1"}
                     2 {::fund/code "fund 2"}}})

(pco/defresolver contract-resolver [{::contract/keys [code]}]
  {::pco/output [::contract/id ::contract/code ::contract/name]}
  (get-in db [:contracts code]))

(pco/defresolver position-resolver [{::contract/keys [id]}]
  {::pco/output [{::contract/positions [::position/units {::instrument/fund [::fund/id]}]} ]}
  {::contract/positions (get-in db [:positions id])})

(pco/defresolver fund-resolver [input]
  {::pco/input [::fund/id]
   ::pco/output [::fund/code]
   ::pco/batch? true}
  (clojure.pprint/pprint
   {:in input
    :res (map #(get-in db [:funds (::fund/id %)])
         input)})

  (map #(get-in db [:funds (::fund/id %)])
       input))

(p.eql/process
  (pci/register [contract-resolver position-resolver fund-resolver])
  [{[::contract/code "contract-1"] [::contract/name {::contract/positions [::position/units {::instrument/fund [::fund/code]}]}]}])
everything works perfectly if db is changed to
(def db {:contracts {"contract-1" {::contract/id   1
                               ::contract/code "contract-1"
                               ::contract/name "the name"}}
          ;;notice that positions are now a vec
         :positions {1 [{::position/units  10
                         ::instrument/fund {::fund/id 1}}
                        {::position/units  20
                         ::instrument/fund {::fund/id 2}}]}
         :funds     {1 {::fund/code "fund 1"}
                     2 {::fund/code "fund 2"}}})
#2021-02-0317:26wilkerlucio@jmayaalv yes, that's a limitation, I think the right approach at this time is to give a better error message#2021-02-0317:27wilkerluciothe issue is that batch needs to be able to navigate, it would be possible to make lists work, but at some processing cost (because I can't update-in a list, but I can in a vector)#2021-02-0317:27wilkerluciocan you open an issue for it?#2021-02-0317:27jmayaalvgot it, yes will do. Thanks a lot!#2021-02-0318:59markaddlemanI'm struggling to surface resolver exceptions in Pathom3. What attr should I provide to pcrs/get-attribute-error to get the internal error exception?
(pco/defresolver throws [{:a/keys [input]}]
  (throw (Exception. "Internal error"))
  {:a/output (str "echoing " input)})

(-> (p.eql/process (-> [throws] (pci/register))
                   [{[:a/input "hello"] [:a/output]}])
    (meta)
    ::pcr/run-stats
    (psm/smart-run-stats)
    (pcrs/get-attribute-error :a/output))   
#2021-02-0319:00markaddlemanUnder the latest Pathom3 commit, get-attribute-error itself returns
{:com.wsscode.pathom3.connect.runner.stats/node-error-type :com.wsscode.pathom3.connect.runner.stats/node-error-type-unreachable
    :com.wsscode.pathom3.connect.runner/node-error #reveal/error{:via [{:type clojure.lang.ExceptionInfo
                                                                        :message "Can't find a path for :a/output"
                                                                        :data {:com.wsscode.pathom3.attribute/attribute :a/output}
                                                                        :at [com.wsscode.pathom3.connect.runner.stats$attribute_error__30270 invokeStatic "stats.cljc" 73]}]
                                                                 :trace [[com.wsscode.pathom3.connect.runner.stats$attribute_error__30270 invokeStatic "stats.cljc" 73]
                                                                         [com.wsscode.pathom3.connect.runner.stats$attribute_error__30270 invoke "stats.cljc" 51]...}
#2021-02-0319:12wilkerluciohello @markaddleman, the issue is that you are looking for the error in the wrong map#2021-02-0319:12wilkerlucioerrors and stats are per entity (each map in the response gets its own)#2021-02-0319:12wilkerluciothis should work:
(-> (p.eql/process (-> [throws] (pci/register))
      [{[:a/input "hello"] [:a/output]}])
    (get [:a/input "hello"])
    (meta)
    ::pcr/run-stats
    (psm/smart-run-stats)
    (pcrs/get-attribute-error :a/output))
#2021-02-0319:12markaddlemanah ha#2021-02-0319:14markaddlemanThx!#2021-02-0319:15markaddlemanOne more thing: Is there a convenient & performant way of checking if any attributes have errors?#2021-02-0319:15wilkerlucioyes, in the run stats, there is a key that says all the nodes with errors#2021-02-0319:15markaddlemanMy desired exception handling policy is to throw an exception if any resolver throws so I'm writing my own query function to handle this#2021-02-0320:13wilkerlucioone thing to consider is that its possible to have a failed node, but still a fully successful response, this can happen if there are multiple paths for something (a OR node), some path fails, other path succeed#2021-02-0320:13wilkerlucioin this case, you will have an error, but still a complete response#2021-02-0320:36markaddlemanUnderstood. In my case, there is no OR but I'll keep this is mind#2021-02-0319:15markaddlemanperfect#2021-02-0319:16wilkerlucioI think you can learn a lot about checking the sources for the attribute-errors-plugin: https://github.com/wilkerlucio/pathom3/blob/master/src/main/com/wsscode/pathom3/connect/built_in/plugins.cljc#L10-L40#2021-02-0319:16markaddlemanperfect! Thank you#2021-02-0323:24markaddlemanWondering what you think about resolver input maps being smart-maps? My app gets a client query, processed by pathom as EQL and many times it is convenient to pass the input around to different functions that might need to compute things for which resolvers already exist.#2021-02-0401:40wilkerluciothat can kill efficiency in a sense, ideally you can just write more resolvers and always be as specific as possible about your requirements, this is the way to maximize the utility of how pathom can optimize your code paths to fulfill data needs#2021-02-0401:41wilkerluciobecause if you give them as smart maps, you delay the point in which you know what you need#2021-02-0401:41wilkerlucioso each time you read, pathom has to go over the planning and running process again#2021-02-0401:42wilkerluciowhile if you do in a single go, pathom can plan once, and with more information makes it more efficient (because it can optimize repeated paths and stuff like that)#2021-02-0401:42markaddleman"each time you read" - This means get operations on the smart map?#2021-02-0401:42wilkerlucioyeah#2021-02-0401:42wilkerluciodoing a single query is much likely going to be faster than lazy loading one attribute at a time#2021-02-0401:43wilkerlucioI had that situation with Pathom Viz, I was using smart maps to compute trace data#2021-02-0401:44markaddlemanFair enough. Although, in my case, I think the only performance difference is Pathom's overhead to compute the plan. No matter what, I'm going to have to compute the result I care about. The only issue is whether the computation occurs as part of a smart-map get or as a direct function call#2021-02-0401:44wilkerlucioI was feeling some perf hit, and switch over to a pre-defined query in EQL, that got around 8x faster (because there was a lot of collection items, in those cases the usage of smart maps to do one attr a time adds up)#2021-02-0401:45wilkerluciowhat I'm telling you is mostly about my perf expectations, but I like to encourage you to play around with a mixed approach, and Pathom totally supports that#2021-02-0401:45wilkerlucioyou can write a plugin to make this happen#2021-02-0401:45markaddlemanOh, that's a good idea. I had planned on each resolver wrapping input in a smart map but a plugin is much more convenient#2021-02-0401:45wilkerluciousing the wrap-resolve hook, there, you can wrap the resolver call by transforming the input in a smart map, them you get what you asked for#2021-02-0401:46markaddlemanPerfect#2021-02-0401:46markaddlemanThis whole conversation brings to mind something I've been wondering about: Given smart-maps, what is the right balance between function calls on maps vs get operations & resolvers#2021-02-0401:46wilkerlucioand I'm curious how this goes, would to learn more about what happens after some time doing that šŸ™‚#2021-02-0401:47wilkerluciofunction calls on maps?#2021-02-0401:47markaddlemanI mean a function that takes a map as input#2021-02-0401:51wilkerluciothat's the design part, depends how much you want to delegate to pathom#2021-02-0401:51wilkerlucioor what "entry points" you wanna give to the system#2021-02-0401:51wilkerlucioI think a very open question, varies a lot#2021-02-0919:57eoliphanti’ve been struggling with that as well lol.#2021-02-0920:05markaddlemanHere is my latest thought: Because Cursive does a much better job of navigating among function defs and function calls, my sanity prefers function calls over smart-map resolvers and keys. It's just much easier to reverse engineer what's going on. Perhaps when I become more familiar with Pathom Viz and other debugging tools, I'll change my mind.#2021-02-0920:05markaddlemanThat's not to say I don't think smart-maps & resolvers have their place... They definitely do. It's just that when I don't have a compelling reason one way or the other, I'll stick to fns#2021-02-0420:53tony.kayHow to I get to the reader ā€œinputā€s in a Pathom 2 wrap read plugin? All I see is env, and I’m not sure where the inputs would be#2021-02-0420:54tony.kaydo I have to do my own ā€œselect-keysā€ on the entity atom for the reader?#2021-02-0421:10wilkerlucio@tony.kay you can't on wrap-read, but you can on ::pc/wrap-resolve#2021-02-0421:11tony.kayAH#2021-02-0421:11tony.kaywrap-resolve!#2021-02-0421:11tony.kaywrong wrapper!#2021-02-0421:11wilkerlucioyup, the wrap-read will wrap the call to the reader chain#2021-02-0421:12tony.kaywow, went too low-level and totally got stuck…so easy#2021-02-0516:07markaddlemanThis code results in an internal guardrails exception in the latest Pathom3. I'm not sure if specifying a key path in ::pco/input is proper usage. In fact, if I trim the path in each of the resolvers to a single key (ie, :ui/query and :pql/query) , pathom is happy. Further, if I remove one of the resolvers, pathom is happy.#2021-02-0517:05wilkerluciofrom your description it seems a bug somewhere, what happens if you disable guardrails? the result is what you expect?#2021-02-0517:07markaddlemanI haven't tried disabling guardrails - I'll give that a shot in an hour or so#2021-02-0615:40markaddlemanWith guardrails disabled, Pathom3 has throws a stack overflow. This is the first many frames#2021-02-0615:50markaddleman(sorry for the delayed reply - I got distracted yesterday)#2021-02-0717:27wilkerlucio@markaddleman fixed https://github.com/wilkerlucio/pathom3/commit/d9a950c96f3fe3115a54679228dbf92c2ca06c9b#2021-02-0717:27markaddlemanFantastic thanks!#2021-02-0717:29wilkerluciothe issue was a recursion in the nested thing, the edge of ui-query->time-range input was the same key as the output (`:query/timeRange`)#2021-02-0717:29wilkerluciothat drover the planner to do an infinite recursion, but fixed now#2021-02-0717:30wilkerluciothanks for testing and bringing those cases to the light šŸ™‚#2021-02-0717:27wilkerlucio@markaddleman fixed https://github.com/wilkerlucio/pathom3/commit/d9a950c96f3fe3115a54679228dbf92c2ca06c9b#2021-02-0613:20dehliHello! I just read the following on https://pathom3.wsscode.com/docs/mutations > Different fromĀ `inputs`, parameters don’t have auto-resolution, they always come as-is, but having this information can help on creating extensions (to support auto-resolution if wanted) and help to understand the system (documentation). Would the extension mentioned be a plugin?#2021-02-0712:02wilkerlucioyes šŸ‘ #2021-02-0613:20souenzzo@wilkerlucio How can I debug the pathom-viz? I'm getting "Response timeout" when connect to some parsers (pathom3) But in network tab, there is no activity#2021-02-0613:22souenzzoI found in terminal output: PayloadTooLargeError: request entity too large#2021-02-0613:23souenzzohttps://github.com/apostrophecms/apostrophe/issues/1291#issuecomment-375928334#2021-02-0616:39wilkerluciothis should had been fixed in a few versions ago, are you on latest?#2021-02-0719:52markaddlemanThanks. I'm switching my production app over to use Pathom3. I'm seeing nice wins with smart-map (primarily by hiding some case switching logic from the clients) . There is a big win by using the new placeholder capability in EQL.#2021-02-0815:51wilkerlucioawesome, can you also check how performance is affected with the migration?#2021-02-0815:51wilkerlucioI wonder how much of the gains I see in my "lab tests" translate to real app performance gains#2021-02-0816:06markaddlemanI can't really do performance comparisons, unfortunately. Performance on the API layer is not really an issue for us since many of our APIs are backed by OLAP-style queries on the database. Given the performance of the DB queries is typically 5 - 15 seconds, the performance of the API shim layer almost doesn't matter#2021-02-0802:50dehliWith pathom3, is there a way to access the options map that I pass into an operation inside of a plugin? I’d like to write a plugin that detects if a spec is passed in the options map and if so, validate the params before invoking the mutate. My thoughts are it could be used like below, and then the plugin would be responsible for validating the spec. If this isn’t possible, I could always wrap the defmutation with my own custom macro to accomplish this. Thanks!
(pco/defmutation my-mutation
  [params]
  {::pco/spec ::my-mutation-spec}
  ;; If this is executed, I know that params fulfills the ::my-mutation-spec
  {::success true})
#2021-02-0802:58wilkerlucioyes you can, its a bit different for mutations and resolvers, but both are accessible#2021-02-0802:59wilkerlucioin the wrap-mutate you get the AST, you can use the :key there to know the mutation name, use that to lookup into ::pci/index-mutations#2021-02-0803:00wilkerlucioa mutations is a record, you can get the options map with pco/operation-config#2021-02-0803:00wilkerlucioin the case of a resolver what you get is the node, the node has an ::pco/op-name, you can use that to find the resolver in the ::pci/index-resolvers#2021-02-0803:01wilkerluciofor convenience, use the helpers pci/mutation-config and pci/resolver-config#2021-02-0803:03dehliawesome, that helps a lot (and is awesome)! thanks for your guidance!#2021-02-0803:16wilkerlucioI would suggest name that something like param-spec to make it cleaner, and better to use your own namespace for it (to make the inference of ownership easier)#2021-02-0803:23dehligood call! i agree that name is cleaner. got it working as well! thanks again!
(p.plugin/defplugin params-spec
  {::p.plugin/id `params-spec
   :com.wsscode.pathom3.connect.runner/wrap-mutate
   (fn [mutate]
     (fn [env {:keys [key params] :as ast}]
       (let [{:keys [params-spec]} (pci/mutation-config env key)]
         (when (and (some? params-spec) (not (s/valid? params-spec params)))
           (throw (ex-info "Invalid params" params
                           {:message (s/explain-str params-spec params)})))
         (mutate env (cond-> ast
                       (some? params-spec)
                       (update :params #(st/select-spec params-spec %)))))))})
#2021-02-0816:46henrikIn Pathom (2), is there anyway to reconstruct env after (or during) a mutation? This is primarily because we need to use a new Datomic DB value after the mutation has run. The pathom-datomic plugin also seems to use an old DB value after mutation is run.#2021-02-0816:56wilkerluciotrying to remember if that works, but try returning a modified env in the key ::p/env in the return of the mutation data#2021-02-0816:58henrikJust saw that here: https://blog.wsscode.com/pathom/#updating-env Thanks! Going to check if this also works for mutations.#2021-02-0817:02avocadeThanks @U066U8JQJ that worked :i_love_you_hand_sign::skin-tone-2:#2021-02-0817:03henrikYep, that worked!#2021-02-0821:21souenzzo::p/env only work on reads or in mutations too?#2021-02-0821:27wilkerlucioworks on reads#2021-02-0821:44wilkerlucio@U2J4FRT2T https://blog.wsscode.com/pathom/#updating-env#2021-02-0821:45wilkerluciono feature for that in pathom 3, maybe that will get add, but I remember having problems of reliability with it, gotta put some hammock time on it#2021-02-0821:46wilkerluciolike in some situations the env could leak out, I wanna Pathom 3 to be safer on that#2021-02-0900:23souenzzo@U06B8J0AJ I work on an pathom app for 1+years now. Now I'm facing some issues because I let my db value at env We are moving the db into input#2021-02-0901:26wilkerlucioyou could also keep db as something mutable in env (as an atom) and update it anytime you see fit (for sync process this should be fine, for parallel not so much)#2021-02-0901:26wilkerlucio@U2J4FRT2T one thing to be careful about db as input is to be sure you are not leaking it out#2021-02-0901:26wilkerlucio(or allowing the users to request teh full db on their queries)#2021-02-0901:28wilkerlucioin Pathom 3 that's quite easy (and efficient) to do, using wrap-map-select-entry, in Pathom 2 its doable, but needs to walk the output to be sure (still easy, but not efficient)#2021-02-0911:05henrik> works on reads Whether intentionally or not, it works on mutations too!#2021-02-0911:05henrikWe're currently doing this after a TX to Datomic,
{…
 ::p/env (refresh-env! env db-after)
 :tempids tempids}
#2021-02-0911:06henrikInserting the updated DB, extracted from the response from Datomic (both for pathom-datomic and our on part of env). It works well.#2021-02-0911:09henrik> you could also keep db as something mutable in env (as an atom) and update it anytime you see fit (for sync process this should be fine, for parallel not so much) We did this at first, and it was not great. Particularly, pathom-datomic didn't follow suit, but also it's fairly unergonomic. It's easy to get wrong.#2021-02-0911:09souenzzowith some cache issues, but it really work
(let [register [(pc/resolver `b {::pc/output [:b]}
                             (fn [{:keys [b]} _]
                               {:b b}))
                (pc/mutation `a {}
                             (fn [env _]
                               {::p/env (update env :b inc)}))]
      parser (p/parser {::p/plugins [(pc/connect-plugin {::pc/register register})]
                        ::p/mutate  pc/mutate})
      env {:b         0
           ::p/reader [p/map-reader
                       pc/reader2
                       pc/open-ident-reader
                       p/env-placeholder-reader]}]
  (parser env `[{(a {}) [:b]}]))
#2021-02-0911:13henrikIn our case, we only needed it on mutations, as queries are the opposite: we absolutely don't want different parts of the query to use different DBs (or, the DB at different times). That can return some real frankensteinish results.#2021-02-0911:14henrikAnd for pagination, we want to be able to keep the DB timestamp stable until the user decides to refresh the query.#2021-02-0920:01eoliphantdid something similar for one of my first plugin attempts, went the atom route#2021-02-0817:31paul931224Hello everyone! I am trying to setup Pathom3 with Pathom Viz. I started here: https://pathom3.wsscode.com/docs/debugging#connect-the-app but I ran in the stacktrace: (short version)
Caused by: java.io.FileNotFoundException: Could not locate com/wsscode/promesa/macros__init.class, com/wsscode/promesa/macros.clj or com/wsscode/promesa/macros.cljc on classpath.
Do I forget to import something?
#2021-02-0817:32wilkerlucio@paul931224 seems like you are in an outdated version of pathom3, can you try the latest commit? (currently: d9a950c96f3fe3115a54679228dbf92c2ca06c9b)#2021-02-0817:35paul931224My mistake, I had the good version a while ago, but somewhere in the process I must have overwritten it. Thank you very much! šŸ™‚#2021-02-0912:39imreHey folks. Given a pathom2 system where I currently have the following resolvers:
{::pc/input #{::aaa/input}
 ::pc/output [::aaa/output]}
and
{::pc/input #{::bbb/input}
 ::pc/output [::bbb/output]}
I want to create a wrapper layer over them that would receive ::api/input and return ::api/output, routing the request to one of the above based on some condition in the input. I have the initial resolver roughly as:
{::pc/input #{::api/input}
 ::pc/output [::aaa/input ::bbb/input]}
(if (use-aaa? input)
  {::aaa/input {}}
  {::bbb/input {}})
But I'm having trouble coercing ::aaa/output or ::bbb/output to ::api/output. How would I go about this? There exist mappings for both aaa->api-output and bbb->api-output.
{::pc/input ???
 ::pc/output [::api/output]}
#2021-02-0913:25wilkerluciohello, I think you can make it work with some aliases, just keep the two resolvers you mentioned first, and add:
[(pc/alias-resolver ::api/input ::aaa/input)
 (pc/alias-resolver ::api/input ::bbb/input)
 (pc/alias-resolver ::aaa/output ::api/output)
 (pc/alias-resolver ::bbb/output ::api/output)]
#2021-02-0913:33imreAren't aliases for the case when the 2 sides of it look the same? In my case there have to be mappings.#2021-02-0913:35imreIt works fine when I only have the ::aaa/output -> ::api/output resolver but when I add the ::bbb/output -> ::api/output one I'm no longer getting the aaa case resolved#2021-02-0913:35imreIt might be something with how my parser is set up#2021-02-0913:39wilkerlucioah, you right#2021-02-0913:40wilkerluciofor the input you should do a routing like you mentioned#2021-02-0913:40wilkerlucioand for output you can use alias#2021-02-0913:43wilkerluciocomplete example:#2021-02-0913:43wilkerlucio
; A adds 10
(pc/defresolver a-output [{:keys [a/input]}]
  {:a/output (+ input 10)})

; B multiplies by 2
(pc/defresolver b-output [{:keys [b/input]}]
  {:b/output (* input 2)})

; lets give evens to A, odds to B
(pc/defresolver input-selector [{:keys [api/input]}]
  {::pc/output [:a/input :b/input]}
  (if (even? input)
    {:a/input input}
    {:b/input input}))

(def register
  [a-output
   b-output
   input-selector
   (pc/alias-resolver :a/output :api/output)
   (pc/alias-resolver :b/output :api/output)])

(def parser
  (p/parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/reader2
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate
     ::p/plugins [(pc/connect-plugin {::pc/register register})
                  p/error-handler-plugin
                  p/request-cache-plugin
                  p/trace-plugin]}))

(comment
  (parser {} [{[:api/input 16] [:api/output]}]) ; => 26
  (parser {} [{[:api/input 17] [:api/output]}]) ; => 34
  )
#2021-02-0913:50imreThank you kindly. It is certainly something with my parser setup then, as the above example does not work with it but it does in your code#2021-02-0913:51fjolne@U08BJGV6E you can also use another helper if you need a custom output mapping instead of 1-to-1 https://blog.wsscode.com/pathom/v2/pathom/2.2.0/connect/resolvers.html#_single_attribute_resolvers or just make everything explicit https://nextjournal.com/a/9KJX2jnmNJBppKKFXHu3mb/edit?token=A7nh9jzVddEhHZTjGinr3Khttps://nextjournal.com/a/9KJX2jnmNJBppKKFXHu3mb/edit?token=A7nh9jzVddEhHZTjGinr3K#2021-02-0913:53imreThank you. Cannot access your second link there, perhaps you sent me the wrong one#2021-02-0913:55fjolneah, sorry, right https://nextjournal.com/a/9KJX2jnmNJBppKKFXHu3mb#2021-02-0913:56imreThank you very much.#2021-02-0913:57imreRunning my resolvers with these parsers works, so I'll need to dig into that. We're using the parallel parser but I haven't familiarized myself with its configuration yet, but it seems the problem lies there somewhere.#2021-02-0914:02fjolnei've had some weird problems with parallel parser, which made me to switch to a regular one; those were hardly reproducible though so I doubt that's it for one thing you can check whether you have pc/open-ident-reader included#2021-02-0914:04imre
::p/reader [p/map-reader
             pc/reader3
             pc/open-join-context-reader
             pc/open-ident-reader
             p/env-placeholder-reader]
#2021-02-0914:04imrelooks like I do#2021-02-0914:07fjolneah, reader3 is experimental, it doesn't work as expected in this case#2021-02-0914:08imreWell, drat. I'll need to look into why we started using it then šŸ˜„#2021-02-0915:35wilkerlucio@U08BJGV6E parallel parser has its quirks, and maybe its a bug related to that, in the reality of Pathom 2 parallel parser is that the overhead it adds is just too high for most applications#2021-02-0915:36wilkerluciounless you have a really big amount of parallelizable things (and are willing to pay some premium on the servers), its usually more efficient (in speed and cost) to stay with reader2#2021-02-0915:36wilkerlucioand about reader3, yeah, that's was an experiment and it is outdate in Pathom 2 (it was the bases for Pathom 3, but in Pathom 3 it evolved a lot since)#2021-02-0915:42imreThank you. I'll check back in the history why we use reader3 but it being outdated is a definite warning sign.#2021-02-0915:43imreWRT parallel, what do you mean by really big amount? In this current case I'm working on there are some library calls that take a significant amount of time and we would prefer to parallelize calls to them.#2021-02-0916:45wilkerlucioI suggest you do a few comparisons and check the final time#2021-02-0916:45wilkerlucioif they are close, stick to serial, since its also easier to debug#2021-02-0917:06imreThank you#2021-02-1017:19fjolne@U08BJGV6E JFYI there’s also async-parser, which allows to return core.async channels and has lower overhead than parallel-parser IIRC#2021-02-1017:20imreThank you, I'm actually experimenting with that šŸ™‚#2021-02-1114:38imreThanks you for your help so far. Mind if I ask what is the recommended reader to be used with async-parser? (and parallel-parser for that matter) I've just been getting into pathom lately#2021-02-1114:57imreLet me rephrase a bit: if I use the async-parser, should is my best choice of reader the async-reader2?#2021-02-1118:55wilkerlucioyes, async-reader2 with async parser#2021-02-1002:15msswhen working with datomic + pathom, is it insane to have multiple entities each have a :`db/id` key, especially considering the case where that :db/id might be the input to other resolvers?#2021-02-1002:15mssmy inclination is that would break and/or be a terrible idea, but not positive on that#2021-02-1003:01souenzzo@mss in pathom2 with pc/reader2, if you use :db/id in every resolver, you may experience some performance issues IDK the current status for pathom3 Anyway, I use pathom2 with datomic, and manually extract each pull from my system. Not a big deal šŸ™‚#2021-02-1003:03mssappreciate the response and insight!#2021-02-1015:55Bjƶrn EbbinghausI think the best solution is not to use :db/id as a key for your API. :db/id identifies an entity in your DB. You can use a key :user/id or :watever-model/id for public use which is another name for :db/id. Or you don't use the db/id at all and generate some other form of public id. (UUID, slugs, ...)#2021-02-1020:02markaddlemanProcessing an EQL query with Pathom3, I get the following error info:
:type clojure.lang.ExceptionInfo
               :message "Insufficient data"
               :data {:required #:data-source{:id {}}, :available {}}
               :at [com.wsscode.pathom3.connect.runner$invoke_resolver_from_node$fn__45209 invoke "runner.cljc" 399]}
It would be helpful if the ex-info data included the name of the resolver(s?) that pathom tried
#2021-02-1020:10wilkerlucioyou have this data in the meta#2021-02-1020:10wilkerluciothere are ways to crunch it from there, or using pathom viz you can see the graph and analyse the parts, have you tried that?#2021-02-1022:39markaddlemanI haven't played with viz yet. Looks like I have a good reason now )#2021-02-1022:39markaddlemanšŸ™‚#2021-02-1022:54wilkerluciogive it a go, connect your parsers and track whats going on in the Requests tab, let me know if have any hiccups in the process#2021-02-1023:16wilkerluciotip: click on the bars on the trace view to open the graph view#2021-02-1118:02markaddlemanI don't have an EQL endpoint for my app. Is there a way to write up Viz as part of my REPL so it can submit EQL as a function call?#2021-02-1118:52wilkerlucioyes, use the connector#2021-02-1118:53wilkerluciowhen you open the app, it has links for how to setup the connector#2021-02-1118:53wilkerlucioboth for pathom 2 and 3#2021-02-1118:53wilkerluciohttps://github.com/wilkerlucio/pathom-viz-connector#2021-02-1119:43souenzzopathom3 placeholders can replace values inside "entity"
(p.eql/process {} `[{(:>/a {:a 1})
                     [:a
                      {(:>/a {:a 2})
                       [:a]}]}])
                    
=> {:>/a {:a 1, :>/a {:a 2}}}
This is a explicit behavior?
#2021-02-1120:35wilkerlucioyup, and there are examples mentioning this specific behavior#2021-02-1120:49wilkerluciofrom docs:
(p.eql/process env
  [{'(:>/bret {::first-name "Bret" ::last-name "Victor"})
    [::full-name
     {'(:>/bard {::first-name "Bard"})
      [::full-name]}]}])
; {:>/bret
;   {:com.wsscode.pathom3.docs.placeholder/full-name "Bret Victor",
;    :>/bard
;    {:com.wsscode.pathom3.docs.placeholder/full-name "Bard Victor"}}}
#2021-02-1216:54kennyWe are running into some serious pathom2 performance issues. We're running 2.3.0. I have APM enabled and can see a crazy amount of request time is eaten up by p/map-reader. It's getting called 40k+ times for a single request. This resolver is returning ~1.5k maps, each with 10+ attributes in them, so I imagine that is what's causing Pathom to spin its wheels. From my resolver, I am adding (with-meta x {::p/final true}) since I remember reading that is how you can force Pathom to not loop through every result attribute. It, however, does not seem to have any impact. This particular query will actually hit 2 resolvers, which could be important. The first returns a list of relevant IDs and the second is a batch? true that returns the 10+ attributes I mentioned earlier. The latter is also the one that wraps its result in ::p/final. Does anyone have an idea on how to fix this?#2021-02-1217:00wilkerluciohello Kenny, p/map-reader is what reads the data from the entity and places in the final map, 1.5k maps is a lot of maps, if you let Pathom do the sequence processing its likely to add that kind of overhead. How long is the request taking?#2021-02-1217:01kenny~10s#2021-02-1217:01wilkerluciothe cases in which I had to deal with large sequences in Pathom 2 was to make the list itself final, so Pathom doenst have to traverse the items, but that would involve pulling the logic out of internal resolvers and manually doing it when you generate the collection, then you make that collection final#2021-02-1217:02kennyThat's what I thought I was doing here with p/final.#2021-02-1217:02wilkerluciosince you told me you are using hte batch, so that means pathom still has to process each entry, the final means "dont process futher", but in this case you already in the items of the collection#2021-02-1217:05kennyNot sure I'm following šŸ™‚ I essentially have this:
(pc/defresolver my-foo-list-resolver
  [env _]
  {::pc/input  #{}
   ::pc/output {:foos [:foo/id]}}
  (let [id-list (get-id-list)]
    {:foos id-list}))

(pc/defresolver my-resolver
  [env xs]
  {::pc/input     #{:foo/id}
   ::pc/output    (get-output-pattern)
   ::pc/transform pc/transform-batch-resolver}
  (let [long-list (get-long-list xs)]
    (with-meta long-list {::p/final true})))
#2021-02-1217:05kennyThis request will first hit a resolver with returns a list of :foo/ids. Pathom will then call my-resolver in batch mode.#2021-02-1217:09kennyDid you mean pc/batch? and ::p/final won't work together?#2021-02-1217:11kennyhttps://github.com/wilkerlucio/pathom/issues/170 is where I am getting my info from.#2021-02-1217:13wilkerluciolets think about how pathom does the process, the resolver first returns a list with ids (like you said), then it runs the batch, in this sense, we can see we are allowing pathom to process the sequence items#2021-02-1217:13wilkerlucioand when the sequence is large, you can see a lot of process going on#2021-02-1217:14wilkerluciothe way in which ::p/final can help you, is to avoid process the list, completly#2021-02-1217:14wilkerluciomy-foo-list-resolver is the one who needs to get the final list ready (and with a final meta), this way pathom is cut out, and you can optimize there#2021-02-1217:15wilkerlucioreturning ::p/final as the meta in a standard resolver response will never work, because that data is merged "sideways" (pathom gets the resolver response, and merges it in the current entity)#2021-02-1217:15kennyOhhh. In this case, I cannot use the pathom resolver "graph" at all.#2021-02-1217:16wilkerlucioin this "sideways" process, its already a pathom thing, so there is nothing to "stop" there#2021-02-1217:16kennyI see.#2021-02-1217:16wilkerluciothe ::p/final must always comes as part of meta of some sub-data (a map or sequence), because them pathom is still going to start process that path, and if it finds ::p/final, it stops at that point#2021-02-1217:18wilkerlucioexamples of valid ways to use ::p/final (writing just the resolver response map for illustration):
; don't process nested list
{:some-large-list ^::p/final [{:col 1} {:col "of"} {:col "items"}]}

; don't process nested map
{:some-large-map ^::p/final {:im "a" :really "long" :map "that" :already "has" :all "the" :data "here"}}
#2021-02-1217:19kennyRight. It sounds like they key is that I can't have Pathom call any nested resolvers though.#2021-02-1217:19wilkerlucioI hope Pathom 3 does better (and benchmarks tell so, but still to see a real app improvement from future portings)#2021-02-1217:20wilkerlucioyup, the final is a scape hatch, but then its up to you to make that part faster, this is usually more useful for simple long maps, where you already have the data upfront and pathom is just filtering the result (than you cut the filtering part)#2021-02-1217:20kennyI've been following Pathom 3 šŸ™‚ Been itching to switch to it but it doesn't sound like it is production ready yet.#2021-02-1217:20wilkerluciomy plan is to announce as a beta soon#2021-02-1217:21wilkerluciothe API is mostly stable (the external ones are, but I'm still reversing myself the right to rename some internals at this point)#2021-02-1217:21kennyWhat if I call the parser from within my-foo-list-resolver? The ::p/final would apply, correct?#2021-02-1217:22wilkerlucioit would, but if you do that you pretty much paying the same price, just in a different place#2021-02-1217:22wilkerlucio(unless you do the calls in some more optimal way, cutting some corners)#2021-02-1217:23kennyHow so? e.g.,
(pc/defresolver my-foo-list-resolver
  [env _]
  {::pc/input  #{}
   ::pc/output {:foos [:foo/id]}}
  (let [id-list (get-id-list)
        long-list (parser env (my-query id-list))]
    {:foos (with-meta long-list {::p/final true})}))

(pc/defresolver my-resolver
  [env xs]
  {::pc/input     #{:foo/id}
   ::pc/output    (get-output-pattern)
   ::pc/transform pc/transform-batch-resolver}
  (let [long-list (get-long-list xs)]
    (with-meta long-list {::p/final true})))
#2021-02-1217:23wilkerlucioyeah, that code would pretty much do the same as just letting pathom do it#2021-02-1217:24kennyWhy?#2021-02-1217:24wilkerluciobecause the parser is going to loop over your entities and process than#2021-02-1217:24wilkerluciothat's the same that was happening before#2021-02-1217:24kennyI guess I'm confused why ::p/final would apply then.#2021-02-1217:25wilkerluciolets come back, my suggestion to use ::p/final depends on you making your own processing for the nested items, that needs to be made faster than how Pathom can do it (which isn't that hard, since you don't have to care about many things that pathom does)#2021-02-1217:26wilkerluciobut if you want to make it faster and optimized, you have to process your collection yourself (without resolvers help)#2021-02-1217:26wilkerluciothat's how ::p/final can be faster, because you let go of using pathom for that portion where you need a faster loop to processa a long sequence (long sequences performance in Pathom 2 isn't that good, Pathom is optimized for complex requests with small output, large outputs pay cost)#2021-02-1217:28wilkerlucioif you have the time, I would love to hear if Pathom 3 can do better, and by how much#2021-02-1217:28wilkerlucioif you are not using many plugins or dynamic resolvers (like graphql stuff), the port to experiment (you can just port this specific part to try out) should be an easy setup to check the difference#2021-02-1217:29wilkerlucioare you using the serial parser, right?#2021-02-1217:30kennyThis is our plugins list
[(p/env-plugin (merge {::p/process-error eql-error-handler}
                 base-env))
 (pc/connect-plugin {::pc/register (get-registry)})
 (p/env-wrap-plugin
   (fn [env]
     (merge env (eql-parameter-lookup (::p/root-query env)))))
 p/error-handler-plugin
 ;; removes ::p/not-found from outputs
 (p/post-process-parser-plugin p/elide-not-found)]
#2021-02-1217:30kennyYes#2021-02-1217:32wilkerluciocool, seems simple, just have to port the eql-parameter-lookup, but I guess for the perf testing you can just avoid it#2021-02-1217:33wilkerluciohere you can see the latest benchmark comparisons I did with Pathom versions: https://blog.wsscode.com/pathom-updates-07/#2021-02-1217:33wilkerlucioI really hope some of those can translate in real gains for user apps#2021-02-1217:33kennyMy guess it it'd "just work." It just adds some eql info to the env:
(defn eql-parameter-lookup
  [query]
  (let [ast (eql/query->ast query)
        params (into {}
                 (keep (fn [node]
                         (when (seq (:params node))
                           [(:key node) (:params node)])))
                 (:children ast))]
    {:cs.eql/root-query-ast ast
     :cs.eql/ident->params  params}))
#2021-02-1217:33wilkerlucio(specially in those cases where CPU gets intensive)#2021-02-1217:34kennyYeah, I read that and was excited to try it. Primary concern is breakage or any bugs with non-prod lib.#2021-02-1217:35wilkerlucioyeah, I feel you, and I can't say Pathom 3 is prod ready yet, I hope that once the beta is out, we can start porting users to go though the "bug crunch" phase#2021-02-1217:36kennySomeone has to go first... šŸ˜…#2021-02-1217:37kennySo ultimately, for places where perf is critical, I cannot use Pathom list processing at all. I must "flatten" the resolvers such that all list data is outputted from a single resolver.#2021-02-1217:38wilkerlucioyeah, if you have long lists, there is a lot going on to process details, and unfortunately the implementation for Pathom 2 ends up adding too much in these cases#2021-02-1217:38wilkerluciothat was surely one of my biggest motivations to make Pathom 3, make it faster to extend the use cases where it can work:slightly_smiling_face:#2021-02-1217:39kennyYep. It looks so awesome. That & the nested map thing look great.#2021-02-1217:41kennyI half inclined to just try 3 since I feel our Pathom setup is very vanilla. Sometimes the bugs can be quite nuanced though, so I am a bit concerned there.#2021-02-1217:52wilkerlucioyeah, the change in processing algorithm is surely scary, because its trading the problem you know for one you don't, and its really hard to know which kind of problems may surface... that said, the only way to find them is trying it out#2021-02-1217:53kennyYep šŸ™‚ Did anything change with how errors are handled?#2021-02-1217:54wilkerlucioyes, that's quite different actually, were you using the plugin to pull errors up? (you could be doing on the client), because the easy error path in Pathom 3 looks more like that#2021-02-1217:55wilkerluciohere you can find information on Pathom 3 and errors: https://pathom3.wsscode.com/docs/debugging#2021-02-1217:55wilkerlucioalso check this built-in plugin: https://pathom3.wsscode.com/docs/built-in-plugins#attribute-errors#2021-02-1218:00kennyOk. Thanks for the help with identifying the problem here. I have a clear path on how to fix it.#2021-02-1218:00kennySuper excited to try Pathom 3!#2021-02-1217:44nivekuilwhere do I catch mutation errors in p3? calling (mutation env ast) in wrap-mutate just returns timing stats#2021-02-1217:53wilkerluciomutation erros come as the mutation result, not sure if you are seeing a bug or missed expectation, can you send an example of what you trying to do?#2021-02-1217:56nivekuil
(pco/defmutation subscribe [_ _]        (throw  (ex-info "Error" {})))  ::pcr/wrap-mutate    (fn [mutation]      (fn [env {:keys [key] :as ast}]        (-> (mutation env ast)            (p/then' (fn [result]                       (log/spy result)                       (some-> (get result key)                               ::pcr/mutation-error                               #(u/log ::mutation-error                                       :exception %))                       result) ))))  result => {app.model.subscription/subscribe {:com.wsscode.pathom3.connect.runner/node-run-start-ms 7.5366440645885E7, :com.wsscode.pathom3.connect.runner/mutation-run-start-ms 7.5366440645885E7, :com.wsscode.pathom3.connect.runner/mutation-run-finish-ms 7.5366441060086E7, :com.wsscode.pathom3.connect.runner/node-run-finish-ms 7.5366441089373E7}}
#2021-02-1217:58nivekuilcan I log the mutation error in that plugin, or should it be somewhere else? the error does appear in the final result returned from from eql/process#2021-02-1218:06wilkerluciohad you tried using catch instead of then to find the error?#2021-02-1218:07wilkerluciothats an interesting example, I have no code testing async extensions on mutations errors, this may require code changes, or some new docs to explain#2021-02-1218:09nivekuilI admit I actually didn't even try catch, but it still seems to be the same behavior: only the p/then path is taken
(-> (mutation env ast)              (p/then (fn [result]                        (log/spy result)))              (p/catch (fn [result]                         (log/spy result)) ))
#2021-02-1218:20wilkerluciocool, I can't look in it right now, but I'll do later today#2021-02-1223:56wilkerlucio@U797MAJ8M was a bug, fixed on master, here is an example to track and log errors for async runners:
(let [env (-> (pci/register
                (pco/mutation 'foo {} (fn [_ _] (throw (ex-info "Error" {})))))
              (p.plugin/register
                {::p.plugin/id
                 'log

                 ::pcr/wrap-mutate
                 (fn [mutation]
                   (fn [env ast]
                     ; running mutation inside p/do! allow us to use catch to capture both
                     ; sync and async errors
                     (-> (p/do! (mutation env ast))
                         (p/catch (fn [e]
                                    ; log error here, and make sure you return the
                                    ; error at the end
                                    e)))))}))]
  @(p.async.eql/process env ['(foo)]))
#2021-02-1300:44nivekuilnice! thanks for the example, didn't know about p/do#2021-02-1300:47wilkerlucioglad to help, also check p/let, I find that super useful#2021-02-1301:51nivekuilI think the last commit has a regression: ast only has the value map of`:params` in it. maybe this should be ast on this line? https://github.com/wilkerlucio/pathom3/blob/1da7457bf65dbcd57cc9a7e98bc7eeee7bf6705b/src/main/com/wsscode/pathom3/connect/runner/async.cljc#L348#2021-02-1301:52nivekuile.g.
::pcr/wrap-mutate    (fn [mutation]      (fn [env {:keys [key] :as ast}]        (log/info ast)        ...))
ast there is equivalent to the old (get ast :params), so key is nil
#2021-02-1303:07wilkerluciooh, sorry, you are right, this was because I also wanted to enable the user to override the output, and for that I changed the place of invocation of it, fixing it now#2021-02-1303:14wilkerluciofixed on master https://github.com/wilkerlucio/pathom3/commit/523dab6c4d9c9c9bc2df323a86b4f4e9833f66bb#2021-02-1218:41mssstruggling to get pathom resolvers to reload using tools.namespace.repl w/ pathom 2.3#2021-02-1218:41mssper the fulcro book, I’ve got my user ns set up something like:
(tools-ns/set-refresh-dirs "src/my_dir")

(def system-atom (atom (server/system :dev)))

(defn start
  []
  (swap! system-atom component/start))

(defn restart
  []
  (swap! system-atom component/stop)
  (tools-ns/refresh :after 'user/start))
#2021-02-1218:42mssmy parser creation seems p straightforward:
(defn make-parser [system-config env-entities]
  (pathom/parser
    {::pathom/env     (merge {::pathom/reader               [pathom/map-reader
                                                             pathom-connect/reader2
                                                             pathom-connect/open-ident-reader
                                                             pathom-connect/index-reader]
                              ::pathom/placeholder-prefixes #{">"}}
                             env-entities)
     ::pathom/mutate  pathom-connect/mutate
     ::pathom/plugins [(pathom-connect/connect-plugin {::pathom-connect/register (concat parser-mutations/mutations
                                                                                         parser-resolvers/resolvers)})
                       pathom/error-handler-plugin]}))

(defrecord Parser [datomic system-config]
  component/Lifecycle
  (start [component]
    (if-not (:parser component)
      (do
        (timbre/log :info {:message "Instantiating parser"})
        (assoc component :parser (make-parser system-config {:datomic-conn (:conn datomic)})))
      component))
  (stop [component]
    (if (:parser component)
      (do
        (timbre/log :info {:message "Destroying parser"})
        (assoc component :parser nil))
      component)))
#2021-02-1218:43mssa new parser is getting instantiated on each call to user/restart and yet if I edit an individual resolver and call user/restart, the resolvers don’t seem to actually be updated with the new code#2021-02-1218:43mssany ideas what I might be missing here? obv I might just be an idiot and this has nothing to do with pathom#2021-02-1320:04markaddlemanI'd like to ensure that all all resolver inputs are smart-maps. I have written this plugin:
(p.plugin/defplugin as-smart-map {::pcr/wrap-resolve (fn [resolve]
                                                       (fn [env node]
                                                         (p.ent-tree/swap-entity! env (fn [m] (if (psm/smart-map? m)
                                                                                                m
                                                                                                (psm/smart-map env m))))
                                                         (resolve env node)))})
When this plugin is activated, retrieving data from the smart-map results in a StackOverflowException. What am I doing wrong?
#2021-02-1403:59wilkerluciohello, I wanna think a bit more about it maybe there is a different extension point for it., but I would suggest to not mix smart maps with in resolvers, you should try to just make the resolver ask for the most direct input you need, and can you always add more resolvers to the game, doing smart maps inside resolvers will make your runs more unpredictable and harder to debug (and probably running slower, given you have to plan more times than if you delegated to pathom to do in a single go)#2021-02-1404:00wilkerlucioand putting a smart map on an entity is a bad way to do it, because then you add smart map overhead to each and every operation that pathom does (and that can be a lot of merges)#2021-02-1404:08wilkerlucioall that disclaimer said, to make it "work", the wrapper would need to be in another place, write now it kinda of go around the "node execution", the resolver itself is a bit more inner, so I'm considering if I should rename the current to something like ::pcr/wrap-run-node, and use the ::pcr/wrap-resolve to run around the actual resolve, this could allow for a more internal manipulation of what goes in the resolver, and what goes out#2021-02-1404:09wilkerluciowould be a breaking change, but I'm still open for that at this stage#2021-02-1414:01markaddlemanThanks for the feedback. I ended up adding the required parameters to the resolver's input. At first, it seemed a little weird because it wasn't the resolver directly that required the parameter but a downstream function which, in another context, was obtaining its parameters from a smart-map.#2021-02-1414:03markaddlemanRight now, I'm leaning towards your opinion: Wrapping a resolver's input as a smart-map is a bad idea.#2021-02-1414:41wilkerlucioin case the distance between input and the calling part starts get apart, one thing to consider is if there aren't more resolvers in between that could add clarity to the process#2021-02-1416:56markaddlemanI don't understand this. How would I place a resolver between a resolver implementation and a downstream function?#2021-02-1417:00wilkerluciothis is hard to explain without a concrete example, can you give a quick version of how you doing this, where the input is farther from the resolver call?#2021-02-1517:38ChicãoHi, guys.. may someone knows how I can debug this, I've call (:y smart-map) but when I saw the errors I got this
:node-error #error{:cause "Insufficient data",
:data {:required {:test/constant-headers {},
:test/access-token {},                                                                                                                                 :test/body {}},
:available #:test{:constant-headers {:User-Agent {},
:Host {}}}},
:via [{:type clojure.lang.ExceptionInfo,
:message "Insufficient data",
:data {:required {:test/constant-headers {},
:test/access-token {},                                                                                                                       :test/body {}},                                                                                                                 :available #:test{:constant-headers {:User-Agent {},                                                                                                                                                                :Host {}}}},
I've print in test/access-toke and test/body inside their resolvers and i saw the data there, and in the env I saw the data, any help is welcome.
#2021-02-1523:18nivekuilis it crazy to want to replace every backend function with pathom resolvers? like, instead of a http-get function you have a resolver that takes a :url and returns a :http-response#2021-02-1523:19nivekuilI guess that example is probably too general to be useful. it would have to be some kind of domain-specific http-response#2021-02-1603:14nivekuiltrying to implement the above, but not sure what's wrong
(defresolver response [{:keys [uri]}]   {::pco/output [::response]}   (p/let [response (http/get uri)]     {::response response}))  (def smart-map (psm/smart-map (pci/register response) {:uri ""}))  (::response smart-map) => nil
with the log line: WARN :com.wsscode.pathom3.connect.runner/invalid-resolver-response - {:com.wsscode.pathom3.connect.operation/op-name app.resolver.uri/response, :response #object[java.util.concurrent.CompletableFuture 0x22b2c2f4 "pending"]} the response does appear to be correctly fetched within the resolver if I log/spy it
#2021-02-1612:05souenzzo@kevin842 you are using an async (returns a CompletableFuture) resolver with a sync runner (smart-maps) For now, as far i know, to async you need to use the "process" with env+query#2021-02-1613:59wilkerlucio@kevin842 you can't use async with smart maps, you need to use the async EQL process for this#2021-02-1615:07nivekuilah thanks. don't see it mentioned in docs, will it be added someday?#2021-02-1615:08nivekuilnvm it is mentioned, but not too explicitly: To async, there is only the EQL interface#2021-02-1615:20ChicãoHi, when I invoke my logout mutation and pass the values to it, inside it I see the namespace/request instead of the value, does anyone know why this happened?
(p.eql/process env
               [`(logout {::request request
                             ::url url
                             ::cookie-store new-c})])
#2021-02-1615:24wilkerlucioits because you are using the backtick to expand, you have to use ~ before the values to get them there#2021-02-1615:24wilkerlucio
(p.eql/process env
               [`(logout {::request ~request
                             ::url ~url
                             ::cookie-store ~new-c})])
#2021-02-1615:33ChicĆ£othks#2021-02-1616:13nivekuilwill pathom ever walk the same node twice? e.g. A->(B | C) C->target B->A' will pathom ever cycle back to A from A'? I have an alias resolver from A'->A but it seems it never walks back to A, to retry the A->C->target path again starting from A'#2021-02-1616:28dehliI believe the default behavior is that it won’t b/c the resolver’s response is cached for a specific input#2021-02-1616:35nivekuilthat's a good point, but A' would have a different value than A (and I wouldn't want it to loop if they were equal), and I think the cache is keyed based on the resolver params?#2021-02-1616:38nivekuiltried setting ::pco/cache? false on everything, no diff#2021-02-1616:44dehliAhhh, so in your example A' would go through the resolver A but it’s a different ā€œidā€ from what the initial A resolver was called with#2021-02-1616:46nivekuilyeah, an alias-resolver creates that path from A' to A, i.e. A with the value of A'#2021-02-1617:19wilkerlucioyes, Pathom may duplicate the same node in different points of execution, this may happen with OR cases#2021-02-1617:19wilkerluciobut no cycles#2021-02-1617:20wilkerluciothe planner never has cycles#2021-02-1617:21nivekuilgot it. I just ended up explicitly specifying the connection. this is super cool.. I think I am actually going to try to replace my entire backend with pathom :p#2021-02-1617:21nivekuilthe i/o parts at least#2021-02-1620:58wilkerlucioyup, I keep doing that, a lot of Pathom Viz UI work is also supported by pathom inside šŸ™‚#2021-02-1616:43ChicĆ£oI had this mutation and when I invoke it, got this error Can't find a path for :async
(pco/defmutation logout [{::keys [request url cookie-store] :as logout-input}]
  (http/post url (assoc request :cookie-store cookie-store))
  logout-input)
how I called the mutation
(p.eql/process env
               [(logout {::request request
                             ::url url
                             ::cookie-store new-c})])
I need to use something like async runner to mutation with http calls?
#2021-02-1616:47dehlipathom3 uses https://cljdoc.org/d/funcool/promesa/6.0.0/doc/user-guide. I’m not sure which http library you’re using but if it returns a promise you can do something like:
(:require [promesa.core :as p])

(pco/defmutation logout ...
  (p/let [response (http/post url)]
    ;; Use response as you'd like
#2021-02-1616:47Chicãonice, thanks for help me#2021-02-1616:47dehliif the lib uses core.async, then you can look into https://github.com/wilkerlucio/promesa-bridges#2021-02-1617:34wilkerlucio@UL618PRQ9 I just noticed the way you are calling the mutation is wrong#2021-02-1617:34wilkerlucioyou removed teh backtick thing#2021-02-1617:34wilkerlucioand the mutation needs to be as data#2021-02-1617:34wilkerlucioin the way you doing, you are invoking the mutation and returning is result as a query#2021-02-1617:35wilkerluciothe code we talked before was right, as:
(p.eql/process env
               [`(logout {::request ~request
                             ::url ~url
                             ::cookie-store ~new-c})])
#2021-02-1617:35wilkerlucioā˜ļø this and what you are doing are not equivalent#2021-02-1617:36ChicĆ£oWow, thanks a lot!#2021-02-1617:41ChicĆ£oit's worked.#2021-02-1618:49nivekuildebugging with pathom viz, I see that a node I expect to succeed has this attached to it:
:com.wsscode.pathom3.connect.runner/node-error  #object[Object [TaggedValue: unknown, #error {  :cause "Insufficient data"  :data {:required {:app.pathom.uri/response {}, :app.pathom.uri/response-url {}}, :available {}}
but scrolling down, I do see valid data in node-resolver-input:
:com.wsscode.pathom3.connect.runner/node-resolver-input  {:app.pathom.uri/response ...   :app.pathom.uri/response-url ...}
#2021-02-1618:52nivekuilnot sure what the cause is.. the nodes go like response -> response-url -> OR -> (node with error | OR | another node)#2021-02-1618:58nivekuilthe node with error is reachable through some queries but not others, even though they should follow the same initial path.. hm. seems I have to explicitly specify another attribute in the query, but I would expect pathom to be able to get to it regardless? should it be necessary to "hint" the planner like this?#2021-02-1621:55dehlican you post what the failing query looks like and the relevant resolvers (pseudocode is fine)#2021-02-1715:25nivekuillet me demonstrate a different, maybe related planner behavior, with viz nodes:#2021-02-1715:26nivekuil#2021-02-1715:26nivekuil#2021-02-1715:27nivekuilthe top picture is with the bottom-left ->view resolver with a higher priority. it takes that path, realizes it doesn't have enough info, and just dies, neglecting the middle response->response-url path#2021-02-1715:27nivekuilbottom picture doesn't set a priority on the bottom-left ->view, it takes the proper path and succeeds#2021-02-1715:27nivekuilit seems like the planner is not capable of backtracking in general?#2021-02-1718:28nivekuiltested the latest commit and it seems both cases I ran into here now work as expected. thanks @U066U8JQJ ! :D#2021-02-1718:29wilkerluciothere was some planning fixes that landed yesterday, I guess they fixed your case too šŸ™‚#2021-02-1718:29wilkerluciothe error handler does backtracking#2021-02-1718:29wilkerlucioif you find any other weird cases, please let me know#2021-02-1718:38wilkerlucio@kevin842 another thing that may be interesting to you, if you look in the meta, in the node-stats key, if there is any error, it will have a key called "nodes-with-errors", this is a more "global" way to find any errors, you can use this a way to ask the question: did any error happen?#2021-02-1716:45ChicĆ£ohi guys I'm trying to reproduce some error, but maybe I've been invoke mutations wrong, my example code is in this link https://gist.github.com/matheusfrancisco/09de37256dd94d8935e4d3be64adf7c9 , but
(pco/defresolver ex11 []
  {::pco/output [:ex.xico/http]}
  {:ex.xico/http (http/post "" {:headers {:Content-Type "application/json"}})})

(pco/defmutation mutation-ex3 []
  {:ex.xico/cc (http/post "" {:headers {:Content-Type "application/json"}})})

(def env1 (->
            (pci/register
              [ex11
               ex12
               mutation-ex3])
            (psm/with-error-mode ::psm/error-mode-loud)))
;; note in line 41 with psm/with-error-mode if I comment #_(psm/...) this line, the mutation will be work. 
;; with this line(41) uncommented the invoke at line 47 return an error, I put in first comment bellow

(:ex.xico/http (psm/smart-map env1))
(p.eql/process env1  [`(mutation-ex3)])
when I had been invoke my mutation i got this error
:data #:com.wsscode.pathom3.attribute{:attribute :async},
       :via [{:type clojure.lang.ExceptionInfo,
              :message "Can't find a path for :async",
              :data #:com.wsscode.pathom3.attribute{:attribute :async},
              :at [com.wsscode.pathom3.connect.runner.stats$attribute_error__25438 invokeStatic "stats.cljc" 73]}],
       :trace [[com.wsscode.pathom3.connect.runner.stats$attribute_error__25438 invokeStatic "stats.cljc" 73]
but, when I comment the (psm/with-error-mode ::psm/error-mode-loud) my mutations call the clj-http/post. may I don't know much about pathom, but should I don't use (psm/with-error-mode ..) actived with mudations?
#2021-02-1717:19nivekuilI wonder if it's better to use a http client that returns CompletableFutures, like https://github.com/gnarroway/hato#2021-02-1717:21nivekuilare you getting that error from the smart map call or the p.eql/process call?#2021-02-1717:21Chicãoat the p.eql/process call.#2021-02-1717:23nivekuilbut the smart map call works? maybe you can't pass the psm/with-error-mode to an env used for eql calls? It's under the smart map namespace so I think it wouldn't do anything for p.eql anyway#2021-02-1717:25Chicãothe smart-map calls works, and if I change my mutation to another resolver it'll be work too.#2021-02-1717:26nivekuiltry making a different env, without psm/with-error-mode for the p.eql call#2021-02-1717:27nivekuilI don't think there's any reason for that to be part of the p.eql env#2021-02-1717:27ChicãoI really don't know what is the problem (initial I was thing that pathon was cache the others http calls , but i really don't know hehe#2021-02-1717:27ChicãoI remove psm/with-error-mode from env and the mutations was called#2021-02-1717:31nivekuilyeah so psm/with-error-mode may be incompatible with p.eql, but does work with mutations#2021-02-1717:37wilkerlucioit shouldn't be, the eql runner just ignores the smart map error mode, it has no relationship#2021-02-1717:37wilkerlucio@UL618PRQ9 I can't follow your line numbers on this example#2021-02-1717:38wilkerluciocan you send a minimal reproduction of the problem? that's the best way to bring some issue#2021-02-1717:38Chicãoi'm trying to reproduce this with a mock server http, but doesn't throw the error..#2021-02-1717:39ChicãoI try to create a minimal example here https://gist.github.com/matheusfrancisco/09de37256dd94d8935e4d3be64adf7c9#2021-02-1717:45Chicão@U066U8JQJ I follow the stack trace into pathom3 namespaces and in this function should have this key :com.wsscode.pathom3.attribute/attribute but it doesn't there
(defn sm-env-get
  "Get a property from a smart map.

  First it checks if the property is available in the cache-tree, if not it triggers
  the connect engine to lookup for the property. After the lookup is triggered the
  cache-tree will be updated in place, note this has a mutable effect in this data,
  but this change is consistent.

  Repeated lookups will use the cache-tree and should be as fast as reading from a
  regular Clojure map."
  ([env k] (sm-env-get env k nil))
  ([{::p.ent/keys [entity-tree*] :as env} k default-value]
   (let [ent-tree @entity-tree*]
     (if-let [x (find ent-tree k)]
       (wrap-smart-map env (val x))
       (let [ast   {:type     :root
                    :children [{:type :prop, :dispatch-key k, :key k}]}
             stats (-> (pcr/run-graph! env ast entity-tree*) meta ::pcr/run-stats)]
         (when-let [error (and (refs/kw-identical? (get env ::error-mode) ::error-mode-loud)
                               (-> (p.eql/process (pcrs/run-stats-env stats)
                                                  {:com.wsscode.pathom3.attribute/attribute k}
                                                  [::pcrs/attribute-error])
                                   ::pcrs/attribute-error
                                   ::pcr/node-error))]
           (throw error))
         (wrap-smart-map env (get @entity-tree* k default-value)))))))
it means something to you, that helps me to explorate this problem? in this part of code
(-> (p.eql/process (pcrs/run-stats-env stats)
                                                  {:com.wsscode.pathom3.attribute/attribute k}
                                                  [::pcrs/attribute-error])
                                   ::pcrs/attribute-error
                                   ::pcr/node-error))
#2021-02-1717:56wilkerlucio@UL618PRQ9 I just copied and ran your minimal example, but I can't see any issues#2021-02-1717:56wilkerlucio
(p.eql/process env1  [`(mutation-ex3)])
=>
#:com.wsscode.pathom3.demos.buzzlabs{mutation-ex3 #:ex.xico{:cc ""}}
(:ex.xico/http (psm/smart-map env1))
=> ""
#2021-02-1717:58wilkerluciowhat is wrong? how can I see it in my end?#2021-02-1717:59ChicĆ£oys, my example doesn't throw exceptions, because I can't reproduce it my self yet. I trying to, I really don't know why this happened#2021-02-1720:52eoliphantHey I’m playing around with pathom-viz (2021.2.1) via connect-env (2021.01.25), request trace is checked, and everything else looks good but traces aren’t showing up. perf meta data are on the result as expected in the repl, just don’t see the trace stufff#2021-02-1721:05wilkerluciousing pathom 2 I guess?#2021-02-1721:05wilkerluciodo you have the trace plugin installed?#2021-02-1721:12eoliphantit’s pathom3, but i do have pathom2 jars, etc. i’m migrating the app, so just created a new ns to start playing with the v3 parser#2021-02-1722:48wilkerluciopathom 3 always have trace, not sure what may be going on, tried following docs from pathom viz connector?#2021-02-1722:48wilkerlucioor using via http?#2021-02-1802:19eoliphantWill dbl check the docs, etc#2021-02-1922:54eoliphantfigured it out, at a high level lol. i created a new project with just p3, worked fine. added the p2 dep, still worked fine. went back to my app, still broken. but i then removed a require in my user ns that was transitively pulling in p2 stuff, and voila! so not sure of the specifics but it’s a p2/p3 thing for sure#2021-02-1922:58wilkerluciogotcha, can you make a repro case?#2021-02-2317:54eoliphantok ugh, it’s happening again lol#2021-02-1809:38jmayaalvWe got asked today the possibility to expose our pathom api as a graphql api. i know this has been discussed several times and there are attempts like https://github.com/denisidoro/graffiti but do you think it would make sense to relay on pathom3's smart maps as lacinia resolvers? so basically we would create the graphql schema define the root queries attaching resolvers that use smartmaps#2021-02-1812:13wilkerlucioI would avoid smart maps for this use case, doing large queries one attribute at a time isnt very efficient, and some queries will pay more than others#2021-02-1812:13wilkerlucioI think there is some work to be done, like Grafitti to improve that story#2021-02-1812:14wilkerlucionot on my radar at this time, but I can help if anyone wants to try that direction#2021-02-1819:37jmayaalvThanks for the input wilker. It's probably something we will need to implement at some point. I can start looking at it, is the graffiti code a good place to start ?#2021-02-1900:29wilkerlucio@jmayaalv cool, I just started this discussion in the project, if anyone would like to bring points on how to make a GraphQL interface out of Pathom, we can concentrate resources here: https://github.com/wilkerlucio/pathom3/discussions/18#2021-02-1819:12markaddleman#2021-02-1819:12markaddleman(please disregard the namespace name - I'm not sure this is a bug šŸ™‚ )#2021-02-1819:14markaddlemanI have a resolver called "dimension-attributes" which can produce an array of raw data which another resolver formats into something - in this example, that formatting has trivial logic in the id-event-level resolver. Finally, I have a resolver field-list which reformats the data into something suitable for the pivot table UI widget#2021-02-1819:15markaddlemanThe dimension-attributes resolver can produce an empty array#2021-02-1819:16markaddlemanWhen the dimension-attributes resolver produces an empty array, the reformatting resolver (field-list) is not invoked and, so, the field-list resolver is not invoked. This results in the wrong data sent to the UI#2021-02-1819:17markaddlemanI've tried various combinations of pco/? but I cannot find the right combination to get the result I'm looking for.#2021-02-1819:17markaddlemanAm I going about this the wrong way?#2021-02-1823:46dehlithe problem is when you return an empty array you’re not returning those keys (`:key`, :some , and :data) so pathom won’t be able to invoke the id-event-level resolver (which is how your field-list’s attribute-id input can be fulfilled). what would you like the response of (:>/input {:key "empty"}) to be?#2021-02-1823:52dehliInstead of returning a vector would this work?
(pco/defresolver dimension-attributes [{:keys [key]}]
  {::pco/input  [:key]
   ::pco/output [{:attributes/dimensions [:key :some :data]}]}
  {:attributes/dimensions (when (not= key "empty")
                            {:key "key"
                             :some "a"
                             :data "b"})})

(pco/defresolver id-event-level [input]
  {::pco/input [(pco/? :key) (pco/? :some) (pco/? :data)]}
  {:attribute/id input})

;; Keep the rest the same
When you return an empty array, there’s nothing for pathom to iterate on which is the real issue with your original code (and why using a combination of (pco/?) probably wasn’t working).
#2021-02-1900:27markaddlemanThanks. I may have oversimplified the real use case. In real life, dimensions-attributes pulls data from a elastic search which may have zero, one or many documents that match a query#2021-02-1900:27markaddlemanI feel like there ought to be a solution on the field-list resolver by using pco/? but I can't find a magic combination to get the results that I want#2021-02-1900:35wilkerlucio@U2845S9KL just did tried here#2021-02-1900:36wilkerlucioI believe this has the change you want:
(pco/defresolver field-list [input]
  {::pco/input  [{:attributes/dimensions [(pco/? :attribute/id)]}]
   ::pco/output [{:pivot-table/fields [:pivot-table.fields/list
                                       :pivot-table.fields/count]}]}
  {:pivot-table/fields {:pivot-table.fields/count (count (:attributes/dimensions input))
                        :pivot-table.fields/list  (mapv (comp (fn [raw] {:pivot-table.field/id raw}) :attribute/id) (:attributes/dimensions input))}})
#2021-02-1900:36wilkerluciotry with this resolver modified, and let me know if that's expected result now#2021-02-1900:36markaddlemanThx. I'll give it a try right now#2021-02-1900:37wilkerlucioops. I just noticed this broken the first example ("not empty")#2021-02-1900:37markaddlemanYeah#2021-02-1900:38markaddlemanI thought I had tried that šŸ™‚#2021-02-1900:38markaddlemanIt's as if pco/? is causing Pathom to give up too early#2021-02-1900:38dehliIs it expected that we’d need to make it optional? I would think that {:attributes/dimensions []} would still fulfill:
[{:attributes/dimensions [:attribute/id]}]
#2021-02-1900:39wilkerlucioits different specifications#2021-02-1900:39wilkerlucioI guess in his case, the list is mandatory, but the some parts of the list content are optional#2021-02-1900:40wilkerlucioPathom will only consider a sub-query valid if the required fields there are fulfilled#2021-02-1900:40wilkerlucioin that sense, the Pathom behavior is correct in the initial example#2021-02-1900:40wilkerluciobecause the "empty" case can't fulfill the item detail#2021-02-1900:41wilkerlucioaltough, this seems a special case up to discussion, over what to do when the collection is empty#2021-02-1900:41wilkerlucioin this case, should the input be considered valid, or invalid?#2021-02-1900:42wilkerlucio@U2845S9KL changing from required to optional and stop working is for sure a bug, going to work on that in a bit#2021-02-1900:42markaddlemanThanks! This is one of the last bits for me to switch completely to Pathom3.#2021-02-1900:43markaddleman(to be clear, this is new functionality - I haven't tried this in Pathom2)#2021-02-1900:43wilkerluciono nested inputs in Pathom 2, you are living the edge here šŸ™‚#2021-02-1900:43markaddlemanlol#2021-02-1900:43dehliI have many comments in our pathom2 code about where we will benefit from nested inputs šŸ™‚#2021-02-1900:44wilkerlucionice, and Im really asking your opinion here#2021-02-1900:44wilkerlucioin case of empty collections, what you think would be the most sane default?#2021-02-1900:44wilkerlucioconsider the input valid, or invalid?#2021-02-1900:45wilkerlucio(currently is invalid)#2021-02-1900:46wilkerlucioand to make a case for valid, the @U2845S9KL code would work without need for optionality on the fields, if valid was the answer#2021-02-1900:47dehliIn my opinion an empty collection is valid b/c it does fulfill the contract. For every item in the collection (zero items) they have an :attribute/id#2021-02-1900:47wilkerlucioyeah, I'm tending to that side too#2021-02-1900:47wilkerluciomost nested inputs are mostly about aggregation, and aggregation on empty collections most commonly expected to be valid#2021-02-1900:47markaddlemanFrom my standpoint, it took a little bit of staring to understand why the empty collection was not triggering the id resolver.#2021-02-1900:48markaddlemanIt would have been more natural if it had#2021-02-1900:48wilkerlucioyeah, ok, expect that to change, soon šŸ™‚#2021-02-1900:51dehliawesome! what happens if some of the elements fulfill the nested input requirement but others don’t? I don’t have a real life example, but just curious#2021-02-1900:55wilkerlucioin that case its considered a match, but some items may have missing requirements#2021-02-1900:55wilkerluciowonder if would be sane to filter it out#2021-02-1900:58markaddlemanFrom my standpoint, it would be sane to filter those out. I conceptualize pathom's behavior as a database JOIN operation. If some attributes don't exist on one side of the join, they aren't in the result set#2021-02-1901:05dehliya, i also think filtering out those missing the required key is sane behavior. if you didn’t want them filtered out i’d imagine you could make the nested key an optional key instead of required#2021-02-1901:16wilkerlucioI agree with you guys#2021-02-1901:17wilkerluciovalid empty inputs just landed on master! @U2845S9KL you may wanna give a try if you still up šŸ™‚#2021-02-1901:17markaddlemanright after dinner. Thanks!#2021-02-1901:42wilkerlucionested optional inputs fixed on master#2021-02-1901:56markaddlemanWorks perfectly!#2021-02-1901:56markaddlemanThank you!#2021-02-1902:07wilkerlucioyou'r welcome šŸ™‚#2021-02-1902:07wilkerlucioand to finish this up today, nested input filtering just landed on master!#2021-02-1902:08wilkerlucioso now in cases of collections and parts of the items are missing the requirement, these items are filtered out#2021-02-1902:08markaddlemanYep. I just updated my code to remove pco/? . Makes it easier to understand, I think#2021-02-1902:08wilkerlucioyeah, and is the correct expression in your case#2021-02-1903:58dehlithat’s awesome! thanks for the quick updates! excited to play around with the changes#2021-02-1900:29wilkerlucio@jmayaalv cool, I just started this discussion in the project, if anyone would like to bring points on how to make a GraphQL interface out of Pathom, we can concentrate resources here: https://github.com/wilkerlucio/pathom3/discussions/18#2021-02-1901:42wilkerlucionested optional inputs fixed on master#2021-02-1902:07wilkerlucioand to finish this up today, nested input filtering just landed on master!#2021-02-1912:59wilkerluciošŸŽ‰ New docs for how nested input filtering behaves: https://pathom3.wsscode.com/docs/resolvers#nested-inputs-filtering#2021-02-1913:01wilkerluciošŸŽ‰ New built-in plugin, now you can easely make mutation params resolve like resolver inputs! https://pathom3.wsscode.com/docs/built-in-plugins/#resolve-mutations-params#2021-02-1913:01wilkerlucio@U0D5RN0S1 if I remember right you were looking for this one a long time ago#2021-02-1916:49fjolnenice! would be great to be able to apply this plugin non-globally to a single mutation in opts map, kinda like augmentation via pco/transform#2021-02-1916:59wilkerlucionot this plugin directly, but you can make a transform for that#2021-02-1916:56royalaid@wilkerlucio Looks like submitting invalid queries can cause the UI to lock up on Pathom Viz#2021-02-1916:57royalaidi.e. [{[:curriculum/uuid <snip>]}]#2021-02-1916:57royalaidWhich is missing the value to the map which should def. error#2021-02-1916:57royalaidbut the UI shouldn’t require a hard reload IMO#2021-02-1917:00wilkerlucioagreed, can you please open an issue at the pathom viz repo?#2021-02-1917:09royalaidhttps://github.com/wilkerlucio/pathom-viz/issues/25#2021-02-1917:10royalaidSide note, thanks for the amazing tool!#2021-02-1919:29dehliCan you specify params for specific inputs in your resolver? I’d like to have a resolver depend on an input with a specific parameter always applied. My current workaround is to create another resolver that always applies the param but was curious if I could do it along with the input.#2021-02-1919:48wilkerluciono support for that#2021-02-1919:51dehlišŸ‘ thanks! i think i can accomplish what i’m trying to do when i switch to pathom3 by using nested inputs along with the filtering of missing keys#2021-02-1920:48markaddlemanRelated to yesterday's discussion - Now my field-list resolver takes two separate inputs. Both of which can be optional. The first query returns the result I expect. The second query should only invoke the dimension-attributes, dimension->attr-id resolvers and field-list resolvers but returns an empty result instead.#2021-02-1921:25dehliI think this is b/c with your second option you only provide :k1 so it seems like it’s impossible to get to :attributes/profiles which is not marked as optional in your field-list resolver#2021-02-1921:27dehliThe sub-key is optional but it still needs a key of :attributes/profiles which pathom cannot get to with your second query since the only resolver that provides :attributes/profiles requires both :k1 and :k2 .#2021-02-1921:28dehliI bet if you updated the field-list input to be this it would work: (pco/? {:attributes/profiles [(pco/? :attribute/id)]})#2021-02-1921:30dehlior another option would be to mark :k2 as optional in your profile-attributes resolver#2021-02-1921:40wilkerlucio@U2845S9KL before I check it closer, did you upgraded Pathom today? because today I made a fix related to nested optional inputs#2021-02-1921:41wilkerlucioone other tip, you can use the arity 3 on process to make your execution result simpler (by removing the nesting under the placeholder):
(p.eql/process env
  {:k1 "not empty"}
  [{:pivot-table/fields
    [:pivot-table.fields/count
     :pivot-table.fields/list]}])
#2021-02-1921:42markaddlemanYes, I'm using head#2021-02-1921:43markaddlemandoh - Thanks for the arity 3 pointer. That simplifies things.#2021-02-1921:46markaddleman@U2U78HT5G
(pco/? {:attributes/profiles [:attribute/id]})
does not pass a guardrails check. I've been meaning to ask if it should
#2021-02-1921:47wilkerlucio@U2845S9KL wrong place, this is how you do it:
{(pco/? :attributes/profiles) [:attribute/id]}
#2021-02-1921:47wilkerlucioI just tested, that got some results with your code#2021-02-1921:48wilkerlucioand the things that @U2U78HT5G said were accurate#2021-02-1921:48wilkerlucioits the k2 missing dep that's making pathom give up on that#2021-02-1921:49wilkerlucioIm quite happy how you are pushing the boundaries, will be even happier if all works as expected XD#2021-02-1921:49markaddlemanNo worries. I sort of expected this when I decided to go P3.#2021-02-1921:49markaddlemanAnd, to be sure, P3 is proving to be a win for me overall#2021-02-1921:52markaddlemanYeah, if I mark :k2 as optional in the profile->attri-id resolver, field-list produces results but it's the wrong results. There should not be profile in the result from field-list because :k2 isn't in the input#2021-02-1921:53wilkerlucioI'm gonna check that, in the meantime, what about optional on :attribute/profiles?#2021-02-1921:54markaddlemanThat works but... (i think I tried earlier and had a problem)#2021-02-1921:55wilkerlucioabout the optional :k2, I believe its behaving correctly, you made that optional, so it runs the resolver and retuns the list#2021-02-1921:57markaddlemanThis yields the correct result:
::pco/input  [{:attributes/dimensions [:attribute/id]}
               {(pco/? :attributes/profiles) [:attribute/id]}]
#2021-02-1921:57markaddlemanHowever, this
::pco/input  [{(pco/? :attributes/dimensions) [:attribute/id]}
               {(pco/? :attributes/profiles) [:attribute/id]}]
produces a guardrails exception
#2021-02-1921:58markaddlemanAnd this ^^ is closer to my real use case#2021-02-1921:58wilkerluciointeresting, I have a guess on why šŸ‘€#2021-02-1921:59wilkerluciook, that seems a simple spec oversight, the result is correct?#2021-02-1922:00wilkerluciospec fixed on master#2021-02-1922:04markaddlemanI haven't tried turning off guardrails. I'll update to newest master and retest in a few minutes#2021-02-1922:06markaddlemanUnder latest master,
::pco/input  [{(pco/? :attributes/dimensions) [:attribute/id]}
               {(pco/? :attributes/profiles) [:attribute/id]}]
produces correct results
#2021-02-1922:09wilkerlucioI usually use guardrails with throw false#2021-02-1922:09wilkerlucioso even when it complains, things still go on#2021-02-2119:30dehliShould throwing an exception from within a wrap-mutate plugin surface as a mutation error? It seems that the error is being swallowed somewhere.#2021-02-2121:58wilkerlucioir should surface as a error value on the mutation response#2021-02-2202:25dehliI think I’ve figured out what was causing my issue. I have a mutation that’s properly surfacing the error (from the plugin) when I call it regularly. However, if I specify keys to return from the mutation it’s resulting in the error not surfacing.
(pco/defmutation mutate [params]
  {::plugins/params-spec (ds/spec ::spec {::id keyword?})}
  {::params params})

(p.a.eql/process registry `[(mutate {::id "fail")])
;; Returns with a ::pcr/mutation-error

(p.a.eql/process registry `[{(mutate {::id "fail"}) [::params]}])
;; Returns {`mutate {}}
I added a failing test showcasing the difference in behavior in case that’s helpful. https://github.com/dehli/pathom3-plugins/blob/main/test/dehli/pathom3/plugins_test.cljc#L25-L33
#2021-02-2212:57wilkerluciohello @U2U78HT5G, I found the issue on the part that filters the output, that was swallowing the error, fixed on master#2021-02-2212:57wilkerlucioon top of that, a few things I like to suggest after looking at your code:#2021-02-2212:57wilkerlucio- No need to provide plugin-id when using defplugin - p.a.eql alias should be used for async EQL, for sync it should be p.eql#2021-02-2213:00dehliAwesome, I’ll pull down latest! Also thanks for the suggestions! That’ll clean up the code!#2021-02-2213:02wilkerluciooh, since you were so fast#2021-02-2213:02wilkerlucioI acutally pushed a bug, but just fixed#2021-02-2213:02wilkerlucioc50d2e75dd084190c58c0e5a599c8e3f1a88d9ef should be fine#2021-02-2213:03dehliPerfect! Will use that SHA šŸ™‚#2021-02-2213:05dehliJust validated that the fix worked as expected. Thanks for the quick fix!#2021-02-2420:44wilkerlucio#2021-02-2501:56LuanHi, I am using pathom 2 and I would like to know if there is a way to reuse a resolver inside another. For example I have a resolver that verifies if a user exists and I want to reuse this resolver inside another resolver or mutation.#2021-02-2502:17kennyDo you need pathom for this? If not, can you pull the functionality out into a function and call that?#2021-02-2512:59wilkerlucio@U01GKAHT163 the description that you send is bit smelly for me, if its a generic fn, like Kenny said, you can extract and just a regular fn on those. But another (maybe possible) way to think is that you should create some new attribute, on a new resolver, and re-use this in your other resolvers (by attribute name, not resolver name)#2021-02-2512:59wilkerluciowithout context of your application is hard to tell which is more appropriated'#2021-02-2516:16Bjƶrn EbbinghausI use a transformer for things like this#2021-02-2516:33mitchelkuijpersSo I am playing around with pathom3 and I noticed when I do a query like this:
[{'([:my/ident 0] {:my-param "10000"})
    [:my/property]}]
I lose the parameter :my-param in the :my/ident resolver
#2021-02-2516:47wilkerlucionot sure if I understand, complete example?#2021-02-2517:02mitchelkuijpers
(pco/defresolver
    my-ident-by-id
    [env _]
    {::pco/input [:my/ident]
     ::pco/output [:my/property]}
    {:my/property (str "my param =" (:my-param (pco/params env)))})

  (p.eql/process
    (pci/register [my-ident-by-id])
    [{'([:my/ident 0] {:my-param "10000"})
      [:my/property]}])
#2021-02-2517:03mitchelkuijpersthis gives me:
(p.eql/process
    (pci/register [my-ident-by-id])
    [{'([:my/ident 0] {:my-param "10000"})
      [:my/property]}])
=> {[:my/ident 0] #:my{:property "my param ="}}
#2021-02-2517:03mitchelkuijpersI expected the string "my param =10000"#2021-02-2517:04wilkerluciook, gotcha, this situation is expected, because the param is at the ident level, when its reading data inside :my/property that params are gone#2021-02-2517:05wilkerlucioyou have two options here, one is to write a plugin to transfer params down (via env preferably), or put the param in the :my/property, as:
[{[:my/ident 0] ['(:my/property {:my-param "10000"})]}]
#2021-02-2517:06wilkerlucioI believe that was the case in Pathom 2 as well, but Fulcro suggested some plugins to make the transfer down#2021-02-2517:08mitchelkuijpersAah ok I will be calling this from fulcro#2021-02-2517:10mitchelkuijpersI'll play around with this tomorrow thnx Wilker#2021-02-2517:11wilkerlucio@U060GQK8U currently I believe you can make it using the ::pcr/wrap-merge-attribute#2021-02-2517:11wilkerluciobut I also feel this one should be use with some care, because this one happens a lot (every attribute merge)#2021-02-2517:11wilkerlucioso I think for your case a ::pcr/wrap-merge-ident would be more appropriate#2021-02-2517:11wilkerluciothis is an easy add, I'll use the opportunity and work this one later today#2021-02-2517:13wilkerlucioand happy to see you again around here @U060GQK8U šŸ™‚#2021-02-2518:15nivekuilyeah fulcro needs this plugin as df/load! can't put params in attr position. this is what I use
{::p.eql/wrap-process-ast    (fn [process]      (fn [env ast]        (-> (assoc env :query-params                   (reduce                    (fn [qps {:keys [type params] :as x}]                      (cond-> qps                        (and (not= :call type) (seq params)) (merge params)))                    {}                    (:children ast)))            (process ast))))}
#2021-02-2518:16nivekuil(basically a quick port of the RAD plugin)#2021-02-2518:59mitchelkuijpersThnx @U066U8JQJ we were never gone! Still using pathom very happy with it!#2021-02-2608:41mitchelkuijpersI still find it very weird we lose the eql parameters I can see it in the planner:
[{:type :join,
                                                                               :dispatch-key :my/ident,
                                                                               :key [:my/ident 0],
                                                                               :params {:my-param "10000"},
                                                                               :query [:my/property],
                                                                               :children [{:type :prop,
                                                                                           :dispatch-key :my/property,
                                                                                           :key :my/property}]}]}] {:com.wsscode.pathom3.connect.planner/nodes {},
                                                                                                                    :com.wsscode.pathom3.connect.planner/index-ast {[:my/ident
                                                                                                                                                                     0] {:type :join,
                                                                                                                                                                         :dispatch-key :my/ident,
                                                                                                                                                                         :key [:my/ident
                                                                                                                                                                               0],
                                                                                                                                                                         :params {:my-param "10000"},
                                                                                                                                                                         :query [:my/property],
                                                                                                                                                                         :children [{:type :prop,
                                                                                                                                                                                     :dispatch-key :my/property,
                                                                                                                                                                                     :key :my/property}]}},
                                                                                                                    :com.wsscode.pathom3.connect.planner/idents #{[:my/ident
                                                                                                                                                                   0]}},
                                                                 [-477873135
#2021-02-2611:39wilkerlucioits a difference in contexts, consider the params you are proving are the "root level", which is where your ident is, but the properties are a level down#2021-02-2611:39wilkerlucioits not lost, but the placement of it is different from what you are expecting#2021-02-2611:40wilkerluciobecause for pathom, when the subquery is running, that param is part of "parent query", the inside has no visibility about it, its just how it works#2021-02-2611:41mitchelkuijpersAh ok#2021-02-2611:41wilkerlucioin a query that are many plans, if you look at the child plan (for the ident sub-query) this param will not be found anywhere#2021-02-2611:41mitchelkuijpersI fixed it now by creating an ident with a map with two ids#2021-02-2611:42mitchelkuijpersBecause the actual problem is that I need two ids to get something#2021-02-2611:42wilkerlucioin Pathom 3, placeholders can take params for multiple inputs too#2021-02-2611:42wilkerluciolike: [{(:>/foo {:p1 21 :p2 21412}) [:bar]}]#2021-02-2611:42mitchelkuijpersYeah that looks cool but we want to call it from Fulcro not sure how good that will work together#2021-02-2611:43wilkerlucioaltough I imagine it wouldnt be so nice for Fulcro, yeah#2021-02-2611:43mitchelkuijpersI love how that works though#2021-02-2601:15souenzzoHow do I get the "resolver data" at https://pathom3.wsscode.com/docs/plugins#pcrwrap-merge-attribute ?#2021-02-2601:23souenzzoI'm using
(let [{:com.wsscode.pathom3.connect.planner/keys [nodes root]} 
      (-> env :com.wsscode.pathom3.connect.planner/graph)
      sym (-> (get nodes root)
              :com.wsscode.pathom3.connect.operation/op-name)])
Can it be done simpler?
#2021-02-2709:15prncHello šŸ‘‹ what is the current pathom (esp. pathom3) + datomic story? I would love to give pathom a go and I’m looking for pointer re: a good place to start i.e. earn about using it with datomic. Is https://github.com/wilkerlucio/pathom-datomic something one can use with pathom3?#2021-02-2711:16wilkerluciohello šŸ™‚ most people doing serious work with this are integrating Pathom with datomic in a more manual way (writing specific resolvers for parts of the attributes)#2021-02-2711:18wilkerlucioI know @U06B8J0AJ and @U06KD64RX played with pathom-datomic, they may be able to give a better user experiencie, I created it most as an experiment, this is part of Pathom Dynamic resolvers, which is the next thing I'll be working in Pathom 3#2021-02-2712:30prncok, will have to experiment then, which is good as well šŸ™‚ thanks @U066U8JQJ, the progress on pathom3 looks amazing, looking forward to seeing what’s next šŸ™‚#2021-03-0109:46henrikWe found pathom-datomic to be a really nice default to get up and running. We use that as a "baseline", in order to not write a resolver for every single attribute in the DB, and then switch to resolvers for things where we need to inject more intelligence.#2021-03-0109:47henrikThough we're currently on Pathom 2.#2021-03-0109:58prncHi! Thanks so much @U06B8J0AJ. This is what I was thinking, starting out with pathom-datomic. Pathom3 looks tempting thought, esp. for a new/early stage project. Those two are incompatible though right? (i.e. pathom-datomic & pathom3)#2021-03-0110:04henrikI'm not sure @U7MHWDLD8, we haven't tried Pathom 3 yet. When you use Datomic, it's crucial that you can update the environment mid-mutation (since when you transact to Datomic, you need to get a new DB from the result, and inject it into the Pathom env in order for subsequent resolvers to use the correct DB for responses). I'm not sure if Pathom 3 has that yet (@U066U8JQJ did take notice of this in a previous thread), but if it doesn't, I'd recommend sticking to Pathom 2 for the time being.#2021-03-0110:10prncAlright! Will need to do some experimenting now I guess but this very helpful to get me started, thank you!#2021-03-0113:14wilkerlucioone thing to also consider is that Pathom 2 datomic support relies on reader3, which I suggest everyone to avoid, its a dropped experiment in Pathom 2, the Pathom 3 port for that should be easy enough, I'm currently working on dynamic resolvers support (starting with GraphQL because its a harder problem and will get more things clear)#2021-03-0113:23henrik@U066U8JQJ Would you suggest that it's a better idea to go with Pathom 3 at this point, and just write the resolvers required, than to go with Pathom 2 + pathom-datomic + reader3?#2021-03-0113:27wilkerlucioat this exact point you couldn't use the Pathom Datomic with Pathom 3, the safe way to go still Pathom 2 + reader2 + manual resolvers. reader3 has issues that were fixes in Pathom 3 already (bad planning in some situations) but not backported (because there is a lot of non Pathom 2 related things on the Pathom 3 impl). But I'm getting confident with the basics in Pathom 3, experiments have been running fine, and Datomic for Pathom 3 is right around the corner (I guess 1 week, 2 tops), Pathom 3 + Datomic dynamic will be probably safer than Pathom 2 + reader3#2021-03-0114:26prnc> Datomic for Pathom 3 is right around the corner (I guess 1 week, 2 tops) that’s some good news, very exciting!#2021-02-2710:08magraHi!#2021-02-2710:22magraThank you für pathom!! I just upgraded from pathom 2.2.31 to 2.3.1. This causes a massive slowdown on my setup. I have a fulcro app that queries for massively nested trees on load and then works with small changes afterwards. I wrapped the parser with time , which reports "Elapsed time: 3122.194217 msecs" with 2.2.31 and "Elapsed time: 429305.222488 msecs" with 2.3.1. The fans start humming. Is this a known issue? Should I start to debug or should I wait for 3.x and stick to 2.231 in the meantime? I am using p/parser, pc/mutate, p/map-reader and pc/reader2.#2021-02-2711:16wilkerluciohello, are you using Guardrails in your project?#2021-02-2711:20wilkerluciolast time I had a long talk with Tony, and we realised in the end the problem is that in this jump Pathom started using guardrails internally, and then when the user has Guardrails on it causes this kind of slowness in development#2021-02-2712:18magraSwitching off :com.wsscode.pathom/trace fixes it.#2021-02-2712:18magraYes, I am using Guradrails.#2021-02-2712:21wilkerlucioso yeah, Pathom now also pays guardrails price, and that makes slow in dev, but prod still the same#2021-02-2712:21wilkerlucioyou can check on your side by turning guardrails off#2021-02-2713:03magrayes, without guardrails timereturns "Elapsed time: 2452.635043 msecs"#2021-02-2713:03magraThank you!#2021-02-2713:04magraFor your fast reply and for your work on pathom!!#2021-02-2713:04magraAnd I love your song šŸ™‚#2021-02-2713:19wilkerluciohahah, thanks!#2021-02-2714:22wilkerluciohello people, a bit of early news, very excited to say that Pathom 3 is now runnable in Babashka scripts! I'm super happy about it, this means we can now use Pathom engine to make scripts that run blazing fast!#2021-02-2714:22wilkerluciothis needs to be merged before it fully works, but if you use git deps and point to my commit there it works https://github.com/fulcrologic/guardrails/pull/21#2021-02-2714:23wilkerlucioSmart Maps currently don't work, given they require Potemkim, which uses a ton of weird features that Babashka doesn't support, I'm investigating if its possible to make a separate simpler implementation that works in Babashka#2021-02-2714:25wilkerlucio(also requires the latest babashka snapshot, that you can download from #babashka-circleci-builds)#2021-02-2819:45Jakub Holý (HolyJak)Interesting. Do you know of any use cases?#2021-03-0109:23jeroenvandijkInteresting indeed. Do you have an example babashka script somewhere?#2021-03-0304:03lilactowndoes pathom handle wildcards in queries?#2021-03-0304:05lilactownI see some references to it in code but I'm not able to precisely understand the behavior#2021-03-0304:26lilactowndoes it just resolve top-level attributes or does it recurse?#2021-03-0315:31wilkerlucio@lilactown in Pathom, wildcard means "give me everything you loaded so far"#2021-03-0315:33wilkerlucioits not gonna trigger any extra resolvers, when a resolver is called, the full resolver response is always merged, but later it gets filtered out to include only keys that the user asks, the wildcard removes that filter on that level#2021-03-0315:33wilkerlucioone way you can see all possibilities is using a smart map, if you datafy the smart map you can see all reachable keys#2021-03-0317:15lilactownhmm ok. so a client passing a query like:
[{[:component/id 0] [*]}]
isn't going to be any different than:
[[:component/id 0]]
?
#2021-03-0317:22wilkerluciocorrect#2021-03-0317:23wilkerluciowhat may be, is something like: [{[:component/id 0] [:component/some-data *]}]#2021-03-0317:32lilactownhmm I see#2021-03-0318:07lilactownthat's very helpful. I wrote that down here: https://github.com/lilactown/autonormal/issues/6#2021-03-0318:13wilkerlucionice! Im also adding info about this in Pathom docs, I totally missed documenting this#2021-03-0318:33wilkerlucioNew docs for Wildcard usage with EQL interface available at https://pathom3.wsscode.com/docs/eql/#wildcard šŸŽ‰#2021-03-0318:33wilkerlucioNew docs for Wildcard usage with EQL interface available at https://pathom3.wsscode.com/docs/eql/#wildcard šŸŽ‰#2021-03-0513:25wilkerlucio#2021-03-0513:39wilkerluciohello everyone, I like to announce I'll start using Github Projects to manage the tasks around my projects. Starting with the Pathom 3 docs, I made this project here: https://github.com/wilkerlucio/pathom3-docs/projects/1 I like to invite you all to comment and vote on the issues, that can really help me prioritise tasks. If you have other ideas, or things you like to see more documentation of, please feel free to use issues there too. If you are not sure and like to discuss, use the https://github.com/wilkerlucio/pathom3-docs/discussions. I'll be soon making the same for Pathom 3, and them for all my projects, as ideas and tasks come in. Any feedback on the process is welcome!#2021-03-0514:22souenzzoWrong title https://github.com/wilkerlucio/pathom3-docs/issues/12#2021-03-0514:39wilkerluciothanks, fixed#2021-03-0613:52wilkerlucioAlso project for Pathom 3: https://github.com/wilkerlucio/pathom3/projects/1#2021-03-0618:43wilkerlucio#2021-03-0623:39ribeloi wrote a minimalistic clone of pathom, for fun and for fame. maybe someone will find it useful https://github.com/ribelo/pathos#2021-03-0700:18wilkerlucioawesome! I think its a great way to understand how pathom works šŸ™‚#2021-03-0700:19wilkerlucioI like ask you about this text on your docs:
one of the main differences from pathom is that pathos doesn’t bother about the code inside and you can easily use let and whatever your heart desires
#2021-03-0700:21wilkerlucionot sure if I understand, because pathom doesn't care about the code inside the resolver, just what it outputs in the end, maybe a misunderstand of some sorts?#2021-03-0709:57ribelo@U066U8JQJ the most annoying thing I encounter when using pathom is having to write in such a way that the first element in resolver is a map#2021-03-0709:58ribeloin pathos I use a simple macro that goes as far into the function as possible, assuming that the last element returned by the function is either a map or a keyword#2021-03-0710:01ribelothis was the first thing that caught my eye as I read the pathom3 documentation#2021-03-0710:28ribelo@U066U8JQJ i created a PR, maybe you will find it useful
#2021-03-0710:35ribeloanother thing that I don't think pathom makes easy is memoization with ttl, and I think it would be super useful. sometimes data changes every certain time and it would be a good idea to only retrieve it when it's changed#2021-03-0710:37ribeloand by the way, thanks not even for a piece of really great software that I use every day, but for changing my perspective on programming and specifically data driven development ā¤ļø#2021-03-0711:29ribeloi take back what i said about memoization, last time I looked in the documentation, i didn't see the section about cache#2021-03-0711:57wilkerluciohello Ribelo, I posted the reasoning for not having deep checks for output in your issue#2021-03-0711:58wilkerlucioabout cache with TTL, the cache system in Pathom 3 is made to be extensible, you can use the TTL from core.cached for example, there is some integration examples at https://pathom3.wsscode.com/docs/cache#2021-03-0711:59wilkerluciothanks for the feedback, always appreciated#2021-03-0712:07ribeloI read an explanation of why deep check is not a good idea and well, you are right. I hadn't thought of that. šŸ˜‰#2021-03-0716:53markaddlemanShould the resolver r be invoked in this example? Intuitively, I think it should.#2021-03-0716:58wilkerluciothe problem is that you are trying to override an already defined key :k, that wont work, once a value is set, pathom wont change it#2021-03-0716:58wilkerlucioyou cant make a resolver to add information nested like that, not a thing#2021-03-0716:59wilkerlucioit probably is getting called, you just don't get the updated result (because :k was already set at that context)#2021-03-0717:01markaddlemanAh, ok. In fact, it is not getting called but no matter. Your explanation makes sense and, I should have realized that what was going on: It turns out that I rely on that behavior in other areas#2021-03-0717:01markaddlemanThanks#2021-03-0819:12markaddlemanThis query fails due to "insufficient data" for the formatted-metric-query resolver even though the unformatted-metric-query resolver succeeds and requires a subset of inputs and produces the necessary output.#2021-03-0819:13markaddlemanThis is my first time using Pathom Viz (very nice tool, btw!). From what I can tell of the plan, it looks like unformatted-metric-query and formatted-metric-query end up as siblings when I think they one should be a child of the other.#2021-03-0819:13markaddleman#2021-03-0819:30wilkerluciošŸ‘€#2021-03-0819:35wilkerlucioseems wrong, will take a closer look later today#2021-03-0906:47wilkerlucio@markaddleman fixed on master#2021-03-0906:49wilkerlucio#2021-03-0914:38markaddlemanThanks!#2021-03-0915:08wilkerluciowere you able to upgrade? I remember you having issues yesterday#2021-03-0915:56markaddlemanThanks for checking in. I'm still having a problem but it's a bit different than yesterday. I'm beginning to suspect something is corrupted in my cursive/intellij setup#2021-03-0915:59wilkerlucioyeah, I can't reproduce the issue, and the CI in other projects is able to download and use the dep fine, so I guess it is something on your setup, but if you find something please let me know#2021-03-0916:17markaddlemanSo, it's not an intellij problem. I created a new project whose deps.edn is
{:deps {com.wsscode/pathom3 {:git/url ""
                             :sha     "beaf0053e906a18d8bc9d96995e9903995c6926d"}}}
When I run clj -X:deps tree I get
org.clojure/clojure 1.10.1
  . org.clojure/spec.alpha 0.2.176
  . org.clojure/core.specs.alpha 0.2.44
com.wsscode/pathom3  beaf005
  . com.fulcrologic/guardrails 1.1.4
    . expound/expound 0.8.5
  . com.wsscode/cljc-misc 2021.03.09
  . com.wsscode/log  1ac6100
    X com.fulcrologic/guardrails 1.1.3 :older-version
  . org.clojure/core.async 1.3.610
    . org.clojure/tools.analyzer.jvm 1.1.0
      . org.clojure/tools.analyzer 1.0.0
      . org.clojure/core.memoize 1.0.236
        . org.clojure/core.cache 1.0.207
          . org.clojure/data.priority-map 1.0.0
      . org.ow2.asm/asm 5.2
      . org.clojure/tools.reader 1.3.2
  . funcool/promesa 6.0.0
  . edn-query-language/eql 1.0.2
    . org.clojure/spec.alpha 0.2.176
    . org.clojure/core.specs.alpha 0.2.44
  . potemkin/potemkin 0.4.5
    . clj-tuple/clj-tuple 0.2.2
    . riddley/riddley 0.1.12
The only weird thing is the gaurdrails entry. Should my deps reference on gaurdrails directly?
#2021-03-0916:19markaddlemanah, on closer inspection, I see that pathom3 depends on guardrails 1.1.4 and com.wsscode/log depends on 1.1.3. I'm guessing this is the issue? I'm going to figure out how to force a version in deps#2021-03-0916:21wilkerlucioit shouldn't be an issue to have then in different versions, it should just pick the latest#2021-03-0916:21wilkerluciobut strange that its listing both on the deps tree#2021-03-0916:21wilkerluciotried bumping clojure as well?#2021-03-0916:23wilkerlucioI got the same output]#2021-03-0916:23wilkerluciobut I think its only saying that this is not used#2021-03-0916:23wilkerluciowhen I just do clj it opens the REPL fine#2021-03-0916:23wilkerluciois that different for you?#2021-03-0916:28markaddlemanOpening a repl is fine but requiring a pathom ns causes a problem:
āžœ  test-pathom3 clj
Clojure 1.10.1
user=> (require 'com.wsscode.pathom3.connect.built-in.resolvers)
Execution error (FileNotFoundException) at user/eval146 (REPL:1).
Could not locate com/wsscode/pathom3/connect/built_in/resolvers__init.class, com/wsscode/pathom3/connect/built_in/resolvers.clj or com/wsscode/pathom3/connect/built_in/resolvers.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.
user=> 
#2021-03-0916:47markaddlemanMore info: I forked pathom3 and downgraded com.wsscode/cljc-misc to 2021.02.27 (based on nothing but a hunch). I can require a pathom3 namespace successfully. I'll look at cljc-misc and see if something obvious pops out#2021-03-0918:16wilkerluciostrange, I still can't reproduce your issue#2021-03-0918:16wilkerluciothis is in a blank folder that only has a deps.edn with pathom3 latest#2021-03-0919:27markaddlemanI wonder if clojars has a corrupted cljc-misc jar? Are you building that locally?#2021-03-0919:28markaddlemanTo refute my own theory: I exploded both versions of cljc-misc locally and they look ok#2021-03-0920:33wilkerlucioI got a repro and fix, check this out:#2021-03-0920:33wilkerlucio
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "d4a3bdb8bfb1a6b8a1b7bdbb9483bdb8bfb1a6a7f999b5b796bbbbbff984a6bbf9e6"}, :content ("[emailĀ protected]")}
#2021-03-0920:34wilkerlucioafter that, it downloaded things and worked again#2021-03-0920:34wilkerlucio(first delete guardrails and cljc-misc, all versions, then started CLJ, at that point I was abro to repro your issue)#2021-03-0920:35wilkerlucioI expected it to download again on a simple clj, it didn't, but it did when I ran clj -Stree#2021-03-0920:35wilkerlucioafter that I got back working as normal#2021-03-0922:14markaddlemanThanks for digging into this! I'm slammed with another task right now but I'll take a look at this tomorrow#2021-03-1018:25markaddlemanI'm just getting back to this. As it turns out, my issue was resolved by rm -rf ~/.gitlibs . Something must have been corrupted in there#2021-03-1018:36wilkerlucioglad to hear its all back šŸ™‚#2021-03-0819:17markaddlemanWhen I try to include the latest pathom3 sha, clj reports the following error:
Error building classpath. Manifest type not detected when finding deps for com.wsscode/pathom3 in coordinate {:git/url "", :sha "96fc43469b4f1049dbbdc375d698e1c64e44de01"}
#2021-03-0819:27wilkerluciostrange, just tested on a new project and worked fine, can you try cleaning the .cpcache and trying again?#2021-03-0819:34markaddlemanI get the same error šŸ˜•#2021-03-0819:58markaddlemanI tried on a fresh computer and got the same results#2021-03-0906:47wilkerlucio@markaddleman fixed on master#2021-03-1518:32markaddlemanWondering what people's thoughts are on parameter value-driven dispatch for resolvers. Here's my use case: I have some resolvers that compute the same schema output but the particular logic depends on an incoming parameter. For example, I need to compute :customer/invoice but the logic for computing the invoice depends on :customer/tier. This wouldn't be much a problem problem except that platinum tier customers need different inputs than gold tier. My naive approach is to have a single resolver with input of :customer/tier and a confusing union of the inputs required for both platinum and gold tiers. Is there a better way of structuring the code?#2021-03-1518:38wilkerluciohave you considered having multiple resolvers for the tier?#2021-03-1518:38markaddlemanEach resolver would return nil if it didn't apply?#2021-03-1518:38wilkerlucioif a resolver fails (by exception, or missing some parts of the data) pathom tries another one#2021-03-1518:38wilkerlucioyes#2021-03-1518:39wilkerlucioand you can use resolver prioritization to force some to go first (in case it matters)#2021-03-1518:39markaddlemanI thought about it as I was typing up my question šŸ™‚ but I was too lazy to try it out.#2021-03-1518:40markaddlemanYeah, I think that will work. Conceptually, it seems somehow cleaner to have this logic as part of the {::pco/input ..., ::pco/output ...} declaration but the approach you suggest will get the job done#2021-03-1520:07markaddlemanWith guardrails turned on, I am getting the following error:
Execution error (ExceptionInfo) at com.fulcrologic.guardrails.core/run-check$fn (core.cljc:76).

com/wsscode/pathom3/connect/planner.cljc:1279 first-common-ancestor's return type
-- Spec failed --------------------

  nil

should satisfy

  pos-int?

-- Relevant specs -------

:com.wsscode.pathom3.connect.planner/node-id:
  clojure.core/pos-int?

-------------------------
Detected 1 error
Without guardrails, I get the following:
Execution error (AssertionError) at com.wsscode.pathom3.connect.planner/merge-missing-chain (planner.cljc:1428).
Assert failed: Error finding ancestor during missing chain computation
ancestor
(plus large stacktrace) Unfortunately, this happens during planning so pathom-viz doesn't show anything interesting. I have isolated the problem down to one resolver and one of its two inputs. If I include the input, I get this error. If I exclude it, the plan is computed without a problem. If I mark the attribute as optional, it also fails. There are probably two (maybe three) resolvers capable of producing this attribute for the given query - but I don't think that's particularly out of the ordinary. At present, I have a fairly large number of resolvers and I'm not sure how to narrow down the problem to create a reproducible case. Any suggestions?
#2021-03-1520:08wilkerluciothis is likely a bug, this is one part of the planning process that tries to find a common node when something depends on things that are spread across the graph#2021-03-1520:09wilkerlucioone thing you could use is the planner debugger, which goes step by step, its currently broken on the website, but that's something I'll get fixed later today (the UI's on the planner page)#2021-03-1520:09wilkerlucioonce working, you should be able to use this: https://pathom3.wsscode.com/docs/planner/#explorer#2021-03-1520:16markaddlemanwonderful. Can you ping me when you have the ui fixed? I'll hop on it and get you the results#2021-03-1606:49wilkerlucio@U2845S9KL fixed for viz elements just pushed up#2021-03-1615:44markaddlemanHi - I loaded up my OIR index and my query into the tool. I'm not really sure what I'm looking at.#2021-03-1615:44markaddlemanWhen I execute the failing query, I get an indication that the final node is unreachable.#2021-03-1615:45markaddlemanHowever, when I change the query to return the problematic attribute, it computes the plan just fine and returns the correct result#2021-03-1615:46markaddlemanShould the "Log Graph" button do something? When I click it, the console displays a (presumably ClojureScript) object but it does not look intelligible#2021-03-1616:30wilkerlucio@U2845S9KL can you check if you are in pathom latest? because the viz uses its own compiled version of pathom to compute the plan there, I wonder if you missing some fix#2021-03-1616:31wilkerluciothe log graph, yeah, was a oversight, because in dev logs fine, but on the site its not usable#2021-03-1616:31wilkerlucioI can fix that to log a pr-str of the graph#2021-03-1616:37markaddlemanI'm using pathom commit 6b2282e6dc4cfabd449cf7837fcefdd1c9b09891 . I'm not sure how to get the version of Pathom Viz but I'm almost certain it's the latest compiled binary from github. btw, the exception that I'm getting is from Pathom, not Viz#2021-03-1621:42wilkerluciook, I think its time to get the stepper to the next level#2021-03-1621:43wilkerluciocurrently using that view from embed has an issue, actually more than one: 1. it runs using the internal pathom, not the client pathom, and that can cause confusion 2. this version only looks at index-oir, which is not complete, it don't contemplate the nested queries (and I believe that may be the case you are hitting)#2021-03-1621:43wilkerluciothe next level is to integrate that properly in Pathom Viz, some sort of button to debug a query, that can leverage all index, and run in your environment#2021-03-1622:29markaddlemanthat would be great#2021-03-2201:38markaddlemanfwiw, I can't reproduce my original problem. The structure of my resolvers has changed a bit since I last saw the problem. I'll keep an eye out#2021-03-1608:44jmayaalvlately we have been thinking about a way to create something like auto discovery or some kind of documentation for params. Now, It’s hard for clients to know what parameters can be used. initially we are just including the parameters as part of defresolver.
(pco/defresolver a-resolover
   {::pco/output [...]
    ::pco/params [...]})
with the hope that later we could generate somekind of docs from it. does this make sense? would it be worthy to include params as metadata in the index?
#2021-03-1612:35souenzzo::pco/params as any other :custom-key that you put in that map will be included on indexes Tools like pathom-viz can already show it.#2021-03-1612:43jmayaalvawesome šŸ™‚ didn’t know that šŸ™‚#2021-03-1612:43jmayaalvthanks!#2021-03-1612:45jmayaalvone more thing šŸ™‚ what’s the best way on pathom,3 to short circuit the execution of a particular attribute? Currently facing some performance issues when there is an optional attribute in the output.#2021-03-1621:41wilkerluciocan you pinpoint what is getting slow? planning? running? have you checked with Pathom Viz?#2021-03-1716:03jmayaalv@U066U8JQJ yes, we know what’s causing the slow down. Imagine the following:
;; spec
(s/def ::a (s/keys :req [:a/code :a/id])
                   :opt [:a/ref] ) ;; ref is optional

;; data
[{:a/id 1 :a/code "x"} {:a/id 2 :a/code "y" :a/ref "y2}]

(pco/defresolver all
   {::pco/output [:all [:a/code a/ref :a/id]]}
 ...some io )

(pco/defresolver by-id
   {::pco/input  [:a/id]
    ::pco/output [:a/code a/ref]
    ::pco/batch? true }
 ...some io )

(pco/defresolver by-code
   {::pco/input  [:a/code]
    ::pco/output [:a/code a/ref]
}
 ...some io )

with the eql
[{:all [:a/code :a/ref]}]
Pathom now executes all the resolvers trying to find an :a/ref for [:a/id 1] if :all is large enough we get slowdowns. so we thought that the solution is to tell pathom to stop trying to search for the attribute after the first resolver is executed as we already know that the attr will not be found.
#2021-03-1716:13jmayaalvso far to avoid the problem what we did is to create a new batch resolver to solve any optional parameter. so we remove :a/ref from the existing resolvers and created a new batch resolver
(pco/defresolver ref-resolver
   {::pco/input  [:a/id]
    ::pco/output [:a/ref]
    ::pco/batch? true }
 ...some io )
not sure however if this ideal
#2021-03-1716:17wilkerlucioI'm still not getting what is the issue here, and what you expect to be different, can you send images from pathom viz where you can highlight what is slow?#2021-03-1716:18wilkerlucio(and in general, if batch is getting faster results, use it)#2021-03-1716:22jmayaalvi will prepare something from viz but going back to the above: when running:
[{:all [:a/code :a/ref]}]
The all resolver would return [{:a/id 1 :a/code "x"} {:a/id 2 :a/code "y" :a/ref "y2}] already at this moment the programmer could know that [:a/id 1] doesn’t have a ref. however pathom will try to find it so it would go and run by-id with inpunt [:a/id 1] and will not find the ref, so it will go an execute next resolver by-code with input [:a/code "x"] because this is not a batch resolver it could execute many times. (one time for each entry that doesn’t have a ref)
#2021-03-1716:24jmayaalvon pathom 2 the first resolver could return
(with-meta [{:a/id 1 :a/code "x"} {:a/id 2 :a/code "y" :a/ref "y2}] {::p/final true})
so the other resolvers wouldn’t be executed
#2021-03-1716:27wilkerluciogotcha, is about the nested thing#2021-03-1716:27wilkerlucioI was confused because you mentioned OR, so I though was something inside the internal processing of a specific graph#2021-03-1716:28wilkerluciook, I think its time to get ::final in Pathom 3, I can do that later today#2021-03-1716:30jmayaalvšŸ™‚ sorry for the confusion.#2021-03-1716:33wilkerluciono worries šŸ™‚#2021-03-1716:42wilkerlucio@U0J6U23FW ^::pco/final [...] now works on Pathom 3 master (also for maps, etc...)#2021-03-1716:43wilkerluciotests with examples at https://github.com/wilkerlucio/pathom3/commit/58b5585bc99a83661a37f6225a61290b647cb810#2021-03-1716:43jmayaalvoh wow you are fast man!#2021-03-1716:52wilkerlucio@U0J6U23FW thank you for the sponsoring! šŸ˜„#2021-03-1613:11souenzzopathom2 had a ^:final metadata that says to parser "do not enter this structure. Not sure about pathom3#2021-03-1819:18markaddlemanI have a small feature request for Pathom-Viz: Can the list of Requests be shown with most recent request at the top?#2021-03-1819:19markaddlemanHappy to make a PR if you're open to it#2021-03-1819:35wilkerlucioI agree, PR is welcome šŸ‘#2021-03-2021:20markaddlemanSo, I'm playing around with the pathom-viz. I'm getting The required JS dependency "react" is not available, it was required by "cljsjs/react.cljs". when running ./scripts/dev-electron . I'm not well versed with the ecosystems. Is this a problem?#2021-03-2021:36wilkerlucioI think you need to run npm install#2021-03-1822:01mssis there an easy way in pathom 2.2.x to return nil for not found keys instead of :pathom/not-found?#2021-03-1822:23jmayaalv@mss the built in elide-special-outputs-plugin would do that.#2021-03-1822:24mssAppreciate the response!#2021-03-2200:30wilkerlucio#2021-03-2200:30wilkerlucio@U2845S9KL#2021-03-2200:31markaddlemanfantastic!#2021-03-2200:31markaddlemanThanks#2021-03-2200:32wilkerluciohope you have fun šŸ™‚, let me know if you have any issues with it#2021-03-2200:33markaddlemanwill do. I'll take a look later this evening#2021-03-2200:31wilkerlucio#2021-03-2210:49imre@wilkerlucio do later pathom-viz releases have anything for pathom2 users?#2021-03-2210:53imreOr, let me rephrase a bit: do you know which is the latest release that useful improvements for teams still on pathom2?#2021-03-2213:36wilkerlucioyou can always use latest with pathom 2, there are a few design tweeks, and a couple versions ago the auto-complete got some improvements that affected pathom 2 too, but other than that, pathom 2 is done, and these new views have to do with the new way pathom 3 process things, so they dont apply for pathom 2#2021-03-2213:43imrethank you!#2021-03-2219:08littleli@wilkerlucio Hi, I'm author of this issue https://github.com/wilkerlucio/pathom-viz/issues/34 You can reach me here on Slack if you want to. Hopefully I can help with something.#2021-03-2220:04wilkerluciocool, thanks, I post some messages on the issue, we can discuss on the thread there#2021-03-2303:18markaddlemanI have a situation where I'm pretty sure Pathom 3 is computing the wrong path involving resolvers at different priorities. I've been playing around with the new pathom-viz plan stepper but I don't know how to interpret it well enough to understand where things are going wrong. If I dump the index here in slack along with the query, will that be enough to track down what's going wrong?#2021-03-2303:41markaddlemanI think I understand the problem well enough to create a repro case.#2021-03-2315:44wilkerluciohello everyone, I like to give a heads up. my computer motherboard bricked, and due to some brazil forced holidays (due to corona) its expected to take 20 days before I get it back, until then I wont be able to do any progress in Pathom or other open source related stuff. I’ll still be responding questions here, but without my computer what I can try is limited. šŸ˜•#2021-03-2321:14littleliHammock time maybe?#2021-03-2316:53paul931224Hello guys! I always used wrap-reload to keep my backend code fresh. What alternative could I use in my code with connection to Pathom Viz? What I want to achieve is not having to restart my clj parser to see the change in Viz. Should I write something to manually reload my namespace with the env and resolvers ?#2021-03-2317:13jmayaalv@paul931224 i normally have a viz namespace with the tracked-env, everytime i change smoething on the resolvers, i have to reload the viz ns, but no need to restart clj. i am sure there is a better way#2021-03-2317:25paul931224Hmm, I'll try that, but I'll still search for the better way. Thank you :)#2021-03-2317:25royalaidI generally end up having to refresh viz with crtl-r to get the changes picked up after reloading the namespace#2021-03-2317:28paul931224For me, refreshing it doesn't reloads the namespace. But when route handling triggers my wrap-reload , pathom viz will pick up the namespace change aswell.#2021-03-2317:25royalaidBut this is also probably me doing something wrong#2021-03-2317:25royalaidSide note I am shadow-cljs and node land which may also affect things#2021-03-2322:43paul931224Do I have to list all of the keys [:account/id :account/email :account/password] in my ::pco/output ? What if my schema changes, do I always have to change the resolver? Even if the logic doesn't change?
(defresolver account-by-id [{:keys [account/id]}]
  {::pco/output [:account/id :account/email :account/password]}
  (filter-by-value :account/id id user-accounts-db))
#2021-03-2323:17souenzzo@paul931224 no. I personally like to add the keywords "on demand" If for example, you have one funcion that returns (f x) => {:a 1 :b 2 :c 3 :d 4 ....} If I'm working on a task that says "display the value of :a on landing page", I will write a resolver
(defresolver my-f [...]
  {::pco/output [:a]}
  (f x))
With this resolver, if someone ask for [:a :b], it wil return {:a 1 :b 2}, because #pathom will call my-f to resolve :a, merge it result to entity. When pathom tries to find b, it's already on entity, then return. If you ask for [:b], it will return nothing {}, because it can't find any resolver that delivers :b If you are working on a fullstack app, where you control the queries, you can be more "lazy", like me If you are writing an public API, you may decide to always declare all the keys.
#2021-03-2323:29paul931224Thank you very much for the detailed answer. It's just enough flexible for my needs.#2021-03-2400:02paul931224I communicate my EQL query in string, for example: "[{[:user/id 1] [:user/name]}]" then I read-string it, and I give to p.eql/process . The problem is when I try to read-string something like:
[`(save-file {::file-path "./file.txt" ::file-content "contents here"})]
then the result is not what I expected. Are there some alternatives for read-string to resolve this?
#2021-03-2400:33souenzzo@paul931224 First, use clojure.edn/read-string. The clojure.core/read-string is a "Lisp Reader" e can evaluate things:
user=> (read-string "#=(prn :exec)")
:exec
nil
user=>
second, ::file-path is not part of edn spec, it's a clojure shortcut. In edn it should be something like :some.namespace-name/file-path If you do (pr-str [::my-kw]) it will result in the "[:user/my-kw]" string, that is a valid EDN. Some custom EDN readers like edamame allow you to read qualified keywords, but I don't think that it's a good idea for an EQL API
#2021-03-2523:08paul931224@souenzzo I tried it both ways now, and I have a problem with the backtick.
(edn/read-string "[`(a 1)]")
Execution error at user/eval158 (REPL:1).
Invalid leading character: `
user=>
#2021-03-2523:09souenzzobacktick also a clojure thing, not a edn thing.#2021-03-2523:11paul931224Yes, and I want to convert a string of a mutation , so I can send it to pathom. Is there any other way? I check out edaname now.#2021-03-2523:11souenzzo(edn/read-string "[(a 1)]") => [(a 1)]#2021-03-2523:13souenzzo
(= (edn/read-string "[(a {:b 42})]")
   '[(a {:b 42})]
   `[(~'a ~{:b 42})]
   (vector (list (symbol "a")
                 (hash-map (keyword "b")
                           42))))
=> true
#2021-03-3003:49paul931224Thank you, I found my misconceptions!#2021-03-2711:44dehliHello! I have a question about pathom2's cache. I have a resolver that returns a vector of items. How can I associate those items with an ident so that I don’t have to look them up again if they’ve already been returned? The basic code looks like:
(pc/defresolver items-resolver
  [_]
  {::items [{::item {::id 0 ::name "foo" ::depends-on nil}}
            {::item {::id 1 ::name "bar" ::depends-on {::id 0}}}]})

(pc/defresolver item-resolver
  [{id ::id}]
  ;; Don't want this to have to be called for the `depends-on` key in my query below
  {::item (get-from-db)})

(pc/defresolver item-normalizer
  [{item ::item}]
  {::id ...
   ::name ...
   ::depends-on ...})

;; Query
[{::items [::name {::depends-on [::name]}]}]

;; Expected result
{::items [{::name "foo" ::depends-on nil}
          {::name "bar" ::depends-on {::name "foo"}}]}
#2021-03-2717:39wilkerlucioPathom doesn't have the concept of normalization like that#2021-03-2717:40wilkerlucioone thing you can do is manually pre-fill the cache for a resolver you expect is going to be called#2021-03-2717:40wilkerluciothe batch works like this too, you have to add a cache entry on the resolver with the exact input and output you want (the input must have only the keys required by the resolver, otherwise the cache will miss)#2021-03-2717:44wilkerluciothis line has an example of pathom caching: https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/connect.cljc#L837#2021-03-2801:14dehliThanks! I think pathom caching can solve what I'm trying to accomplish. Will give that a go!#2021-03-2920:25dehliWhat do the e and p stand for in the snippet you pasted? I’m running into some issues so my guess is that I’m not exactly supplying the data properly. Currently I have:
(p/cached env ['my.resolver/symbol {:user/id "foo"} {}]
  {:user/output "bar"})
where {:user/id "foo"} is the input into the supplied symbol and {:user/output "bar"} is what it returns. If I don’t wrap the output in a channel I get an error with take!
#2021-03-2920:27dehliI see that e is basically input which matches what I’m doing and p are params so it must be my output that is formatted incorrectly which also matches the behavior I’m seeing with a take! error#2021-03-2920:48dehliI think I got it! I needed to use p/cached-async instead! šŸŽ‰#2021-03-2819:51azHi all. Wondering if anyone is using Pathom to pull from an sql type db or does this really only make sense with datalog? Wondering if Pathom builds an optimized query of some sort or does it run the resolvers body as it builds the plan?#2021-03-2821:23souenzzoCheckout #walkable#2021-03-2821:34az@U2J4FRT2T thank you, checking it now#2021-03-2920:49souenzzoHey @U0AJQJCQ1 I mostly work on #datomic -like apps But I use #pathom as my default "data source" Any data that I need: an database access, an http requests, s3, etc... I access throught #pathom resolvers. So, if I worked with SQL, i would use pathom/resolvers to it too. #walkable tries to resolve a harder problem: expose an SQL DB as an graph API (via pathom). Even if you don't use #walkable, you can use #pathom , writing resolvers manually with your own queries.#2021-03-2920:49souenzzoTL;DR: you don't NEED #walkable to use #pathom with #sql, but when #walkable be ready, it will be an awesome lib.#2021-03-2920:50az@U2J4FRT2T thank you for the explanation. Just out of curiosity, are you ever building restful endpoints as well with Pathom or mostly eql or graphql endpoints?#2021-03-2920:58souenzzoIt's easy to create #pedestal endpoints, for example, getting data from pathom. You can use [(:my-ns/my-name {:pathom/as :name})] to get the value as {:name "example"} I do mostly like this: https://github.com/souenzzo/eql-as#real-world-exmaple There is also an internal tooling on company that checks if the query that we run on the endpoint "conforms" with the swagger/openapi spec.#2021-03-2921:22azThank you @U2J4FRT2T looks great#2021-03-3009:18fjolnehi, i've been playing with p3 recently and found a weird behaviour on nested inputs resolvers: they mess up indicies in some way, both not working by themselves and making other resolvers invisible. here's a repro: http://nextjournal.com/fjolne/pathom3-nested-input-issue#2021-03-3009:20fjolnei tried looking through pathom viz snapshots, but the logs did't make much sense: some existing relationships are marked as unreachable when I add nested inputs resolver#2021-03-3010:34roklenarcicHas anyone tried to expose a bunch of resolvers as a GraphQL interface? I see examples where GraphQL is called by pathom, but never the other way around… It could be useful when you need to expose it to other frameworks/languages#2021-03-3010:41jmayaalv@U66G3SGP5 For pathom2 there is https://github.com/denisidoro/graffiti for pathom3 there is still nothing available yet, there is an open discussion with ideas to achieve something similar https://github.com/wilkerlucio/pathom3/discussions/18#2021-03-3012:34roklenarcicInteresting… thanks#2021-03-3018:02fjolnealso got a problem with p3 batch resolvers, created issues for both: https://github.com/wilkerlucio/pathom3/issues/29 , https://github.com/wilkerlucio/pathom3/issues/30#2021-03-3020:29wilkerlucioawesome, thanks, I'll have a look as soon as possible#2021-03-3020:49dehliis there anything special needed to get batch resolvers working with the parallel parser in pathom2? i’m running into some issues when i follow the docs here: https://blog.wsscode.com/pathom/v2/pathom/2.2.0/connect/resolvers.html#_n1_queries_and_batch_resolvers.#2021-03-3020:59wilkerlucionothing special about parallel parser there, what issue you are facing?#2021-03-3021:07dehliHmmm, I’ll keep digging. A query i have that normally takes ~20s (without batch) ends up having a lot of Insufficient resolver output in the response. The basic resolver shape looks like this (which works when I have batch? false )
(if (sequential? input#)
  (->> input#
       (mapv get-fn#)
       promesa.core/all
       cljs.core.async.interop/<p!
       (mapv out-fn#))

  (->> (get-fn# input#)
       cljs.core.async.interop/<p!
       out-fn#))
#2021-03-3021:21wilkerlucionote this may be an error in your batch output#2021-03-3021:21wilkerlucioit must match exactly the input in number of items and order, otherwise you will have broken results#2021-03-3021:29dehlithanks! that’s a helpful tip!#2021-03-3112:04dehliAfter digging into this more it seems to be due to there being two resolvers that can get to the same key. With batch it keeps spitting out insufficient resolver output (referencing the resolver that shouldn’t be used) and without batch, pathom can appropriately get to the right resolver.#2021-03-3112:17dehliwhat’s strange is my basic query is below. The first one appropriately returns :user/id however the second one has the insufficient resolver output which must be for :user.session/started-at since it takes as input :session/id and :user/id.
;; Works with batch and non-batch
[{[:user/id "foo"]
  [{:user/staff 
    [:user/id]}]}]

;; Doesn't work with batch, works with non-batch
[{[:user/id "foo"]
  [{:user/staff 
    [:user/id
     {(:>/session {:session/id "bar"})
      [:user.session/started-at]}]}]}]
#2021-03-3121:04dehliFor pathom3, is there something that needs to be done to have it run AND nodes in parallel? when looking at pathom viz i see that they’re still executing sequentially. Thanks in advance!#2021-03-3122:04wilkerlucioparallel support is not implemented yet in pathom 3#2021-04-0115:37mokrHi, I’m very early in my decision process in regards to moving from Luminus/re-frame to Fulcro/Pathom. Currently reading and watching to get an overview. I regards to Pathom, I understand that you can have many resolvers that are able to provide the same key, and that the sequence you list resolvers in acts as a priority. But what happens if a resolver just might be able to provide a given key, but can’t guarantee it? Will Pathom try another resolver to fill in the blanks? The context for my question is log analysis. Analysing a handful of log lines for a given session/scenario might or might not provide some piece of information. Can Pathom capture this scenario? Can you state e.g: ::pc/output [:x :y :z], only get :x :y back and have :z provided from another resolver event after running this resolver that stated that it could provide :z?#2021-04-0115:48wilkerluciohello šŸ™‚ yes, pathom does backtrack when there are multiple options, so if some path can't fulfill the data, pathom will try another one#2021-04-0115:51mokrThanks, that sounds good. ā€œBacktrackā€ might be the magic word I did not search for when trying to find the answer. šŸ™‚#2021-04-0116:42wilkerluciomy bad there, this part is not properly documented#2021-04-0505:25zhuxun2Was it intentional that when using the error-handler-plugin, errors thrown by mutations are not added to ::p/errors at the root level?#2021-04-0505:25zhuxun2This makes it difficult to detect whether there is an error thrown by any mutation.#2021-04-0511:11wilkerlucioyou can check for all symbols in the response, and check if they are errors, mutations keys are always symbols#2021-04-0614:04dehliIs there a way to have similar functionality to ::pf.eql/wrap-map-select-entry via pathom2 plugins?#2021-04-0614:08wilkerlucio::p/wrap-read is similar#2021-04-0618:36jjttjjHave there been any more thoughts/discussions on supporting streams/subscriptions in pathom since https://github.com/wilkerlucio/pathom/issues/98 does pathom3 change anything in this regard?#2021-04-0618:37wilkerlucionot at this point, you can implement something about it around Pathom, its the usual issue with any streams: how to detect what changed? who should be notified?#2021-04-0618:38wilkerluciocurrently I’m focusing on getting Pathom 3 basics to work reliably (I still have some classes of problems to figure out there)#2021-04-0618:39jjttjjthanks!#2021-04-0712:13jjttjjI've started trying to implement something where I have attributes refer to a particular websocket connection which has a resolver which initializes the websocket as well as a core.async/mult ("assigned" to another attribute). Then I assign other "Stream types" to an attribute and give them a resolver which both send a message to the websocket to start receiving messages (if that stream type hasn't been created yet) and tap the mult, and wrap the new channel in another mult. In this way I could have a graph of core.async mults that mirrors the pathom graph, and I just query for these mults and tap them in my app. I imagine I will have to manage my own cache with this approach. Are there other obvious reasons this is a bad idea? Has anyone used Pathom to manage stateful components like this?#2021-04-0712:19Alexander KaiserPathom with GraphQl and fulcro 3 Hi, first of all, thank you very much for that fantastic library! I am absolutely fascinated by it. Is there currently an recommended way to use fulcro 3 with pathom and a graphql data source? (preferably via a graphql remote). - The ways mentioned in the pathom 2 tutorial are only supported by fulcro 2 and it seems to be a bit more complicated to port them to fulcro 3 (com.wsscode.pathom.fulcro.network has some dependencies that don't seem to exist anymore in fulcro 3) - pathom 3 looks absolutely awesome but has no graphql helper at the moment I am new to pathom and fulcro. Sorry if I am missing something obvious.#2021-04-0716:19Chris O’DonnellHere's an example project that integrates fulcro 3 with a graphql API via pathom 2: https://github.com/codonnell/crudless-todomvc#2021-04-0721:14Alexander KaiserThank you very much, this helps a lot.#2021-04-0715:31lilactown@alexander.kaiser I would recommend asking in #fulcro, as it sound more topical and people there may be able to help you with the fulcro integration better#2021-04-0721:14Alexander KaiserThanks for pointing this out. I will post future questions regarding this in fulcro channel.#2021-04-0813:01aratareHi there. I currently have a parser tied to my app’s life cycle and would like to know if pathom parsers have any cleanup to do, or if I can safely discard it and create a new one whenever I restart my app. Thanks in advance šŸ™‚#2021-04-0813:58souenzzo@rextruong yes, you can safely discard/create parsers as many as you need/want The parser may hold the "cache", which in general, is safe to discard.#2021-04-0813:59aratareLovely. Thanks for a lot @souenzzo šŸ™‚#2021-04-1111:37Aleksander RendtslevOut of curiosity: Have anyone done any experiments with Pathom outside of Fulcro? (I love Fulcro but I’m experimenting a little with re-frame and reagent these days).#2021-04-1512:27Vincent CantinI did use Pathom while experimenting on the Vrac library.#2021-04-1114:29lilactownwe've started using it a bit at work as just any other API endpoint. we fetch data from it using re-frame and put it in a datascript db#2021-04-1115:41frankitoxThat's pretty cool. You use re-posh?#2021-04-1118:45lilactownwe don't. we just put a datascript db in the re-frame app-db#2021-04-1115:48souenzzo@aleksander990 i use pathom with hiccup to generate SSR pages Also use pathom to implement openapi endpoints#2021-04-1416:20lokirrationalisHi there! Nested inputs is something only Pathom 3 will support, right? Or has it been added to Pathom 2 and I just can’t figure out how to do it?#2021-04-1416:39wilkerluciohello, yes, this is a Pathom 3 only feature#2021-04-1416:51lokirrationalisOkay thanks!#2021-04-1521:13cjsauerIs it necessary to specify ::pc/output of a resolver? I’ve noticed that if I don’t query for any keys that exist in the ::pc/output vector, then my resolver stops working (all keys are :not-found).#2021-04-1522:15wilkerlucioyes, that’s required, there are a few cases in which Pathom can infer the output, but only very simple cases, for the most part you need to define the output#2021-04-1522:54cjsauerAh right. It took me too long to realize, but without the output you wouldn’t know which resolver to actually call…#2021-04-1521:18cjsauerBut if I query for even one key that matches what’s in ::pc/output, now it’s able to find keys that weren’t declared ahead of time.#2021-04-1522:15wilkerluciothis happens because pathom just merges the result, and them you kind get it by accident#2021-04-1522:16wilkerluciothink that pathom uses ::pc/output to generate the attribute index, if you try to lookup for something and its not there, pathom will make it not-found#2021-04-1522:17wilkerluciothis is how you can get the ā€œwork by accidentā€ case:
(pc/defresolver x []
  {::pco/output [:a]}
  {:a 1 :b 2})

(parser [:a :b])
#2021-04-1522:17wilkerluciothe :a attribute made pathom call x, which had the full result merged (`{:a 1 :b 2}`), now, when looking for :b it sees its already on the entity data, so it works#2021-04-1522:17wilkerluciobut if you remove :a from this query (as : (parser {} [:b])), it wont, because :b isn’t indexed#2021-04-1821:40ribelois there any magic way to extract a value from the map returned by eql if there is only one kv?
(-> (p.eql/process env {:db/id 1} [:name]) :name)
this way seems horribly suboptimal
#2021-04-1901:38wilkerlucionothing native, but you can create a helper around that receives a single attr#2021-04-1912:49souenzzo
(let [{:keys [name]} (p.eql/process env {:db/id 1} [:name])]
  name)
#2021-04-1821:41ribelowhen pulling one thing out, creating a smartmap seems even worse#2021-04-1902:44wilkerlucio#2021-04-1902:49markaddlemanThanks for the update!#2021-04-1914:49JAtkinsThis is AWESOME! I’ve been thinking about this + the visualization style for a long time, and it’s really neat seeing it actually used somewhere. Makes me even more interested in working on tooling to simplify these kinds of tools :)#2021-04-1909:41Matheus MoreiraHello! Recently I started playing with Pathom 2 and now I have a situation where I have 3 resolvers that produce the same output (:tournament/full-name) but require different inputs and I would like to give them some kind of priority or an order by which they should be tried. Is it possible to do it?#2021-04-1914:41jmayaalvPathom 3 has support for resolver prioritization https://pathom3.wsscode.com/docs/resolvers#prioritization not sure if available on pathom2.#2021-04-1915:14Matheus MoreiraThanks, I’ll take a look.#2021-04-1916:37wilkerluciothere is a way to do that in Pathom 2, you can set on the env the key ::pc/sort-plan, in this case you must provide your own function to sort the plans, you can find the default impl here: https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/connect.cljc#L621 that said, its usually not trivial to get a specific resolver order from that, that’s why in Pathom 3 it was made different#2021-04-1918:53Matheus MoreiraIn my case I have a tournament and its full name property can be resolved by 3 different resolvers depending on which relationship the tournament has with one of organizer, series, and season. Tournament full name is only the tournament name if it is connected to a organizer; tournament + series names if connected to a series; tournament + series + season names if connected to season ( organizer 1:n series 1:n season). I would like to define the order as prioritizing tournament-season, then tournament-series, then tournament-organizer resolvers.#2021-04-2023:04wilkerluciogotcha#2021-04-2023:06wilkerluciomodeling wise, I think of this as first having 3 different attributes, one for each name path, the important part is just to be able to talk about them as different things#2021-04-2023:06wilkerluciothem, another resolver can ask for the 3 of them, and decide the order based on the available options#2021-04-2023:09wilkerlucioin Pathom 3 you can use a resolver with 3 optional inputs and make the choice there, in Pathom 2, since there are no optional inputs, what you can do is make a resolver that depends the most broad of the options (the case most easy to access, so it just enough for the engine to trigger it), and inside of the resolver you can call the parser to fetch the other 2 options#2021-04-2023:11wilkerluciosomething like:
(pc/defresolver select-name 
  [{:keys [parser] :as env} {:keys [name-opt1]}]
  {::pc/output [:name-choice]}
  (let [{:keys [name-opt2 name-opt3]}
        (parser env [:name-opt2 :name-opt3])]
    {:name-choice
     (or name-opt3
         name-opt2
         name-opt1)}))
#2021-04-2115:27Matheus MoreiraAh, nice! I’ll try to implement it in my code base. By the way, good to know that you are in the Clojure Brasil group. šŸ™‚#2021-04-2107:32Mark WardleHi all. A style question really. Is it reasonable to use defmutation to define an action that doesn’t really change state? I’d like to leverage pathom as the principal API across multiple backend services including operations such as search or login. I guess the latter might change state - eg audit trail - but search is idempotent. What’s the ā€˜best’ way?#2021-04-2112:40wilkerlucioyes, no problems using mutations like that, I see mutations as ā€œcommandsā€. while attributes are about getting some piece of data, mutations are about telling something to happen, its ok if they dont change state#2021-04-2112:45Mark WardleThank you Wilker!#2021-04-2112:47wilkerlucioaltough search is the most gray area here, because its asking for data more than asking a command, but Im not the police, do whatever works for you :)#2021-04-2117:25prncHi šŸ‘‹ A small question about attribute nesting. I came across this today, trying out pathom3.#2021-04-2117:26prnc
(pco/defresolver foo [{input :input}]
    {:foo
     {:msg (str "Foo: Hello, " input)}})

   (pco/defresolver bar [] ;; no dependencies
     {:bar "This is bar"})

   (def env
     (pci/register [foo bar]))

   (p.eql/process env
                  {:input "World"}
                  [{:foo [:bar]}])

   ;; => {:foo {:bar "This is bar"}}
   
   ;; BUT with...

   (pco/defresolver bar [{foo :foo}] ;; depends on :foo
     {:bar "This is bar"})

   (p.eql/process env
                  {:input "Hello"}
                  [{:foo [:bar]}])
   ;; => {:foo {}}
#2021-04-2117:49wilkerlucioPathom is doing the expected stuff in this example#2021-04-2117:49wilkerlucioon the second case, where :bar depends on :foo, at that context you asked :bar, there is no :foo#2021-04-2117:49wilkerlucioPathom looks at each entity level#2021-04-2117:50wilkerluciothe parent entity has :foo, but that’s not where you are asking for it#2021-04-2117:50wilkerlucio
(p.eql/process env
                  {:input "Hello"}
                  [:foo :bar])
#2021-04-2117:50wilkerlucioin this case ā˜ļø they are siblings, so :bar can see :foo, and that’s valid#2021-04-2117:50wilkerluciobut you can’t ask for parent attributes#2021-04-2117:26prncI was trying to ā€œfurther resolveā€ attribute by pulling from db (datomic). Additional data would be nested in what I already have like :bar in the example. But this doesn’t seem to work when :bar depends on the parent key (:foo here). Nesting works fine when :bar has no dependencies. Is there a way around this or maybe there are good reasons for this and it’s the intended behaviour ;)?#2021-04-2117:52wilkerlucioyes, it is intended behavior#2021-04-2117:52wilkerluciodependencies are across the same entity, and with Pathom 3 there is support for nested requires (goes down), but never going up (looking at parent)#2021-04-2117:53wilkerluciofor Pathom, each map is an indepent entity, if we pulled data from all parents would be just crazy šŸ˜›#2021-04-2117:54wilkerlucioif a parent information is intended to be used on the children, the resolver that provides the children must pass it forward#2021-04-2117:56prncHaha, fair enough! Need to get my intuition around how things work in pathom a little stronger. Initially I was thinking it’s similar in philosophy to spec/rdf w/ global, ā€œstrongly namespacedā€ attributes#2021-04-2117:56prnc…and then resolvers sitting between them, i.e. edges: I have A and want B#2021-04-2118:03wilkerlucioit is, one thing to help is remember to isolate the context in one entity, the parent example, is like this: A has :foo, which is an entity (lets call it B), now I navigate to entity B and ask :bar from it, now why I can’t find :foo in B?#2021-04-2118:04wilkerlucioany time you navigate to a new map, is like navigating to a new entity in the graph#2021-04-2118:04wilkerluciomakes sense?#2021-04-2118:07prncThanks @wilkerlucio — just for context of this, to make it slightly more practical: I was wondering how one could replicate datomic pull behaviour in which e.g. in pathom say I get {:media/uri {:db/id 2344534545}} from some resolver and the request to pathom is [{:media/uri [:uri/host]}] and :uri/host could be resolved based on that :db/id if requested like above. So in my mind the resolver returning {:uri/host} would depend on the parent of {:media/uri ,,, } , no?#2021-04-2118:09wilkerluciono, :uri/host should depend on :db/id#2021-04-2118:09wilkerlucioso using a :db/id you fetch the entity and read the :uri/host from it#2021-04-2118:11prncI see, that makes a lot of sense.#2021-04-2118:19prncThanks so much! Now let me experiment some more and get a better understanding before I tire you too much with all that 😜#2021-04-2118:57wilkerluciono worries, have fun šŸ™‚#2021-04-2117:42markaddlemanI'm a heavy user of pathom3 and I've run into a number of bugs related to @wilkerlucio latest post https://blog.wsscode.com/pathom-updates-09/ . I don't know but I suspect you are running into a planner bug. fwiw, the following resolver will produce a result
(pco/defresolver foo-bar [{_ :input}]
  {:foo {:bar "This is bar"}})
#2021-04-2117:55wilkerluciohey @U2845S9KL, curious, did you tried upgrading to the version after the post? did you still see these kind of bugs after?#2021-04-2117:55wilkerlucio(and to clarify, in the case of prnc, its not a bug)#2021-04-2117:56markaddlemanFunny you should bring that up. I just tried upgrading to the latest commit and ran into some problems. I was just about to ask you if the latest commit contained fixes to the shared attribute problem that you describe in your post šŸ™‚#2021-04-2117:58markaddlemanIn short, I have an attribute that is used by a couple of resolvers that share a parent child relationship. These parent resolver is probably three levels deep in the larger planning tree.#2021-04-2118:00markaddlemanIf I remove the common attribute from the input of one of the resolvers, the plan is generated. However, if the attribute is in the input of both resolvers, no plan in generated#2021-04-2118:00markaddlemanI don't have a good way of creating a small reproducible case#2021-04-2118:01wilkerlucioI also did one fix yesterday related to nested queries#2021-04-2118:01wilkerluciowith the new planner I expect to have none (or very few) bugs related to flat queries#2021-04-2118:01wilkerluciobut I still need to get the generators to make nested cases, them the confidence on those can increase as well#2021-04-2118:03markaddlemangotcha. For now, I am working with an older commit and several of my resolvers must perform p.eql/process explicitly rather than depend on inputs. It's a good enough approach for my needs šŸ™‚#2021-04-2118:04markaddlemanIf it would be helpful, I'd be happy to screenshare my code so you can see firsthand the problem with the latest commit. Maybe do that after you get the generators to make nested cases and you have a chance to work out those bugs.#2021-04-2117:46markaddleman@prnc ^^#2021-04-2215:29prncAs I understand p.eql/process in it’s output is silent about Exceptions. thrown in resolvers? Is checking the meta data, the recommended way of checking for errors atm? E.g.
(-> process-result
      (meta)
      ::pcr/run-stats
      ::pcr/node-run-stats
      ::pcr/nodes-with-error)
Or are there other error handling strategies?
#2021-04-2215:36wilkerluciocheck the attribute errors plugin: https://pathom3.wsscode.com/docs/built-in-plugins#attribute-errors#2021-04-2215:37prncAh nice, thanks! I’ve read through the debugging page but I didn’t get to plug-ins!#2021-04-2215:59henrik@wilkerlucio I have to say though, Pathom 3 seems really snappy. We wrapped all resolvers in Honeycomb logs, and played around with the different parsers/resolvers in Pathom 2, along with batching. Then we threw Pathom 3 into the mix (and immediately got problems due to the well-known batching problem) Bottom line, Pathom 3 without parallelism or batching is just as performant for our particular use case as Pathom 2 with parallelism, batching, & tweaks, if not better. So many kudos on the great work, but also, why is this so? šŸ˜„ Does Pathom 3 produce more clever plans?#2021-04-2216:32wilkerluciohello Henrik! glad to hear about these benchmarks, you make my day happier šŸ™‚ about how, its because Pathom 3 goes much more strait to the point than Pathom 2. Pathom 2 still follows the same parser/readers abstraction that were created by David Nolen in the first versions of Om.next. by that time, connect ideas were inexistent. later when I got to the connect idea on Pathom 2, it was an addition to it, so it has to accomodate itself there. in Pathom 3, instead of connect having to ā€œgo aroundā€ the readers interface, connect is the main thing in Pathom 3, and anything else needs to go around that. This allows me to optimize in such a way that wouldn’t be possible in Pathom 2 the new planner thing also helps, Pathom 2 keeps planning (and in case of errors, replanning) all the time while it runs. Pathom 3 model plans once per entity, and that thing is cached, so all the traversing code to figure things out can be isolated, and by the time Pathom 3 is ā€œrunningā€, it can just go traversing the plan nodes, never needs to replan And most important, I started Pathom 3 with the previous experience about what was bad about Pathom 2, that surely helps me avoid some pitfalls#2021-04-2308:51henrikFrom where I'm sitting, it certainly looks like Pathom 3 is calling the resolvers fewer times than Pathom 2 did, which might account for the performance: it seems to be doing less work. This seems to be the case even though you recently turned off some optimizations, from what it sounds like. I also noticed that I could severely suboptimize Pathom using batching. I don't remember if this was Pathom 2 or 3, but when I turned on batching it seems like it ended up calling more resolvers (though, in parallell, due to how I implemented the batching). From this it seems like it really isn't worth batching too aggressively, unless you know that the resolver is going to quite expensive to run per individual call.#2021-04-2713:38wilkerlucioabout the batching too aggressively, that’s correct, its not worth, batch adds some overhead of its own to happen, so unless the batch cost can come with a good difference from individuals, the individuals will be faster. the future solution for this will be parallel parsing, since it will be able to handle multiple items without relying on batch, that should be good for those cases (and no need for extra configuration, like batch needs)#2021-04-2808:53henrikYeah, I've noticed that it's hard to get any I/O to Datomic faster with batching, since it's not really "I/O", and Datomic caches aggressively.#2021-04-2221:12littleli@wilkerlucio I have a progress on windows build / release process of pathom-viz with github actions. stay tuned for PR.#2021-04-2221:38littlelihttps://github.com/wilkerlucio/pathom-viz/pull/37#2021-04-2221:41littlelitrick is, you have to use shell: powershell and once you're down this rabbit hole you have to have two run builds. One for MacOS and Linux, and separate one for Windows.#2021-04-2221:41littleliifs are ugly, but since the matrix is small its probably acceptable.#2021-04-2222:01wilkerluciomerged, I’m doing a few design adjusts in the Requests tab#2021-04-2222:01wilkerluciothem I’ll make a tag so we get the windows release on the main repo šŸ™‚#2021-04-2222:05littlelišŸ¤ž#2021-04-2222:17wilkerlucio:hand_with_index_and_middle_fingers_crossed: https://github.com/wilkerlucio/pathom-viz/runs/2414822943?check_suite_focus=true#2021-04-2222:26wilkerlucioits up!Ā šŸ˜„Ā https://github.com/wilkerlucio/pathom-viz/releases/tag/v2021.4.22#2021-04-2222:30littlelišŸ‘ šŸŽÆ#2021-04-2222:31littleliCongratulation sir! Your application runs on one more platform šŸ™‚#2021-04-2222:32wilkerluciowithout requiring the user to compile itself šŸ™‚#2021-04-2222:32wilkerlucioI’m updating the docs in Pathom 3 as well, and making the announcement text, will be up in a min
#2021-04-2222:26wilkerlucioits up!Ā šŸ˜„Ā https://github.com/wilkerlucio/pathom-viz/releases/tag/v2021.4.22#2021-04-2222:40wilkerlucio#2021-04-2223:59littlelias I promised to you once. I added Pathom-viz to scoop bucket and it can now be installed with scoop installer from the command line.#2021-04-2713:38wilkerlucioabout the batching too aggressively, that’s correct, its not worth, batch adds some overhead of its own to happen, so unless the batch cost can come with a good difference from individuals, the individuals will be faster. the future solution for this will be parallel parsing, since it will be able to handle multiple items without relying on batch, that should be good for those cases (and no need for extra configuration, like batch needs)#2021-04-2314:52prncWhat is the best way to allow resolvers to access user provided global data? Something like e.g. pbir/constantly-resolver or global-data-resolver (if understand correctly), but user provided. E.g.
(p.eql/process (pci/register [top-level-resolver])
                 {:user-data "Hello"}
                 [{:top-level [:user-data]}]) 
So it’s available to all resolvers regardless of level. Examples I’ve come across are all through modifying the env which I guess I don’t want to do on every request, right? Does that make sense?
#2021-04-2314:56wilkerlucioits ok to modify the env on every request, the global things can go there šŸ‘#2021-04-2314:56wilkerlucioenv access is more direct, I usually put things like DB connections there for example#2021-04-2314:57wilkerlucioyou can also make a resolver, which makes it extensible, but also more expensive and less predictable (compared to env which is always a simple key read)#2021-04-2314:59prncNice! That’s one of my use cases, but that can be considered static. The other thing I had in mind was some user identifier that can be used in many places in the graph but may be ofc different for each request and is provided through the initial data. So I guess I will just add that to env instead, thanks!#2021-04-2408:14wilkerlucioyes, user identifier is a common thing that gets filled in the env for each request#2021-04-2603:47Chris RosengrenHi, trying out pathom 3, is this possible? Consider two different APIs providing the same IDs for some data, is there an automatic way of merging two output lists (by ID or composite attributes?): i.e: (defresolver get-names [] {::pco/output {::names [::id ::name]}} {::names [{::id 1 ::name "Foo"} {::id 2 ::name "Bar"}]}) (defresolver get-ages [] {::pco/output {::ages [::id ::age]}} {::ages [{::id 1 ::age 40} {::id 2 ::age 45}]}) I would like to write a query such as below, and have the results grouped by ID automatically, is this possible currently, and if not is my only option creating a third ::people resolver and manually group-by ::id (p.eql/process (pci/register [get-names get-ages]) [{::people [::id ::age ::name]}])#2021-04-2604:36wilkerluciohello Chris, it is possible, but the path you are taking here makes things a bit harder because the results are wrapped in lists, its easier if resolvers are at the entity level, so instead of get-names a get-name will be easier, same for get-ages, the simplest path would be something like:
(def names-by-id
  {1 "Foo"
   2 "Bar"})

(def ages-by-id
  {1 40
   2 45})

(pco/defresolver get-name [{::keys [id]}]
  {::name (get names-by-id id ::pco/unknown-value)})

(pco/defresolver get-age [{::keys [id]}]
  {::age (get ages-by-id id ::pco/unknown-value)})

(pco/defresolver all-people []
  {::pco/output [{::people [::id]}]}
  {::people (mapv #(array-map ::id %) (keys names-by-id))})

;; run query

(p.eql/process
  (pci/register [get-name
                 get-age
                 all-people])
  [{::people [::id ::age ::name]}])
#2021-04-2604:43Chris RosengrenHi Wilker, thanks for the response, the problem is get-names and get-ages are processed responses from an API that I don't control (that only returns items in bulk).#2021-04-2604:44wilkerluciothere are some tricks you can do in this case, let me write an example for you#2021-04-2604:49wilkerluciook, here is an attribute dance you can do to handle this case:
(defn api-names []
  {:names [{::id 1 ::name "Foo"}
           {::id 2 ::name "Bar"}]})

(defn api-ages []
  {:ages [{::id 1 ::age 40}
          {::id 2 ::age 45}]})

(pco/defresolver names-bulk []
  {::pco/output [::names-index]}
  (let [{:keys [names]} (api-names)]
    {::names-index (->> (coll/index-by ::id names)
                        (coll/map-vals ::name))}))

(pco/defresolver get-name [{::keys [id names-index]}]
  {::name (get names-index id ::pco/unknown-value)})

(pco/defresolver ages-bulk []
  {::pco/output [::ages-index]}
  (let [{:keys [ages]} (api-ages)]
    {::ages-index (->> (coll/index-by ::id ages)
                       (coll/map-vals ::age))}))

(pco/defresolver get-age [{::keys [id ages-index]}]
  {::age (get ages-index id ::pco/unknown-value)})

(pco/defresolver all-people []
  {::pco/output [{::people [::id ::name]}]}
  {::people (:names (api-names))})

;; run query

(p.eql/process
  (pci/register [names-bulk
                 get-name
                 ages-bulk
                 get-age
                 all-people])
  [{::people [::id ::age ::name]}])
#2021-04-2604:50wilkerlucioits kinda what I sent before, but now has extra steps to load from a bulk list and index that in mid-process#2021-04-2604:50wilkerlucioso this way you can access as if they were entity level things#2021-04-2604:52wilkerlucio(just fixed a bug in the code I sent last)#2021-04-2604:53wilkerlucio(`coll` is alias for com.wsscode.misc.coll, which is included with Pathom 3)#2021-04-2605:05Chris RosengrenMuch appreciated, thanks, also really like using the library in general#2021-04-2701:44wilkerluciohello people, I just fixed the embed visualizations in the planner docs of Pathom 3, in case you checked recently and found it broken, its fixed and up now šŸ™‚ https://pathom3.wsscode.com/docs/planner. Also, these are more recent visualizations strait from a new thing that I’m calling Pathom Viz Embed, this is something you can use to embed Pathom Viz in web renderes, currently I’m using on the site and now in the Pathom 3 docs. Docs for the viz embed are not available now, but you can find full examples in the sources for Pathom 3 docs: https://github.com/wilkerlucio/pathom3-docs/blob/master/src/main/com/wsscode/pathom3/docs/components.cljs#2021-04-2704:24wilkerlucioFor anybody playing with Pathom 3 and batches, I just pushed some fixes at version fe29d9d3ec89a0321bf20b41a4f2d161f715b80d, if you had trouble, please try this version#2021-04-2704:27wilkerlucio@U06B8J0AJ#2021-04-2709:16henrikGreat @U066U8JQJ, will check it out#2021-04-2718:54wilkerluciothanks @U06B8J0AJ! Just out of curiosity (and in case you tested šŸ™‚, if you do your batch attempt, does it get faster or slower than non-batching?#2021-04-2721:22henrikIt definitely seems slower I'm afraid. This probably comes down to the fact that when the resolver isn't batched, it is called 70 times for a particular query (each call resolves in about 0.4ms, ~28ms out of a total of 0.1s for the entire Pathom run). When batched, however, the resolver is called with 525 entries, making the overall Pathom query balloon to about 0.3s.#2021-04-2721:23henrikThis is just a quick comparison, I can throw Criterium at it to get more reliable numbers if it'll help.#2021-04-2721:40henrikSo,
{'(count inputs) 525
 '(count (distinct inputs)) 70}
Where inputs is the arguments that the batched resolver is called with.
#2021-04-2721:41wilkerluciogotcha, yeah, for fast resolvers like this is expected that batch isn’t going to be faster#2021-04-2721:42wilkerluciobatch will shine better in case of hard IO that can be batched (looking up some external entity, avoiding n+1)#2021-04-2721:43wilkerluciobut I dont get why you have so many more calls in batch mode, it could be related to the deoptimized graph, but still, if things are not failing, it should be the same number of calls#2021-04-2721:43wilkerluciofrom the 525 entries, some of those share the same inputs? thats how it reduces to 70?#2021-04-2723:41wilkerlucio@U06B8J0AJ I just made a change that will deduplicate same inputs on batch, can you please try again on ca9ea82ef92f8113148b39b884639b2a9e25a29a?#2021-04-2805:51henrikCool! Will do.#2021-04-2805:56henrik> from the 525 entries, some of those share the same inputs? thats how it reduces to 70? Exactly. I'm pulling a highly interconnected graph, so there's a lot of redundancy. Let's say I'm asking for entities A, B, and C. If they're all connected to entity D, it ends up that I get D three times. When not using batching, it seems that Pathom is really good at deduplicating this, so that it only asked for D once. When using batching, it seems like each individual time D was required (once for A, B and C, each, a total of 3 times), this was forwarded to the resolver, resulting in a lot of redundancy in the batched collection (in the example, 3 calls to get D instead of 1 call to get D, which is then reused). I'll check out the change and see if this is no longer the case.#2021-04-2806:17henrik@U066U8JQJ I'm seeing no duplication when using batching now šŸ‘#2021-04-2818:49bbssI am returning tempids from one of my mutations with pathom3 but noticed that it's not returning them. I thought that was working earlier but now I am rather confused. Is there a pathom3 equivalent of
::p/env     {::pc/mutation-join-globals [:tempids]}
#2021-04-2821:11wilkerlucionot yet, but that’s something I’m interested in making work#2021-04-2821:11wilkerlucioI think we need a new plugin entry for the planner to make this work#2021-04-2821:13dehliI solved a similar problem using ::pf.eql/wrap-map-select-entry Here’s a link to the code which I think could work (unless the plugins have changed since I wrote that code) https://github.com/dehli/pathom3-plugins/blob/main/src/main/dehli/pathom3/plugins.cljc#L9-L38#2021-04-2821:46wilkerluciothat’s a valid option šŸ‘#2021-04-2821:46wilkerlucioI was playing with one that can be more performant, by manipulating the AST before the whole process happens#2021-04-2821:47wilkerlucio
(defn mutation-join-globals [globals]
  {::p.plugin/id
   `mutation-join-globals

   ::p.eql/wrap-process-ast
   (fn [process]
     (fn [env ast]
       (process env
         (update ast :children
           #(mapv
              (fn [{:keys [type children] :as ast}]
                (if (and (= type :call)
                         (seq children))
                  (update ast :children into (map (fn [k] {:type :prop :dispatch-key k :key k})) globals)
                  ast))
              %)))))})

(pco/defmutation foo []
  {::pco/op-name 'foo}
  {:value   2
   :tempids {1 "meh"}})

(let [env (-> (pci/register
                [foo])
              (p.plugin/register (mutation-join-globals [:tempids])))]
  (meta (p.eql/process env [{'(foo {:bar "baz"}) [:value]}])))
#2021-04-2903:32bbssThank you both! ā¤ļø#2021-05-0402:38wilkerlucio@bbss walking the code I remembered another simpler option for this:
(let [env (-> (pci/register
                [foo])
              (update :com.wsscode.pathom3.format.eql/map-select-include
                coll/sconj :tempids))]
  (p.eql/process env [{'(foo {:bar "baz"}) [:value]}]))
one thing to consider there, is that it will affect not just mutations, but any entity that has the :tempids key
#2021-05-0402:46bbssThanks! I can see that being useful.#2021-05-0402:38wilkerlucio@bbss walking the code I remembered another simpler option for this:
(let [env (-> (pci/register
                [foo])
              (update :com.wsscode.pathom3.format.eql/map-select-include
                coll/sconj :tempids))]
  (p.eql/process env [{'(foo {:bar "baz"}) [:value]}]))
one thing to consider there, is that it will affect not just mutations, but any entity that has the :tempids key
#2021-04-2914:12wilkerluciofor Pathom 3 users, a planner fix just landed on master, related to partial unreachable paths on attributes with multiple paths, example user report: https://github.com/wilkerlucio/pathom/issues/192, fixed on Pathom 3 master 30f62a10ee78d678821a769ba15dece314d3dfc5#2021-04-2916:05prncHow one would deal with ā€œwritesā€ inside of resolvers, if at all? Context: In the process of resolving EQL query, data is pulled from a third-party API—I would like to both save it to db and return to client, what’s the cleanest way to do it with pathom3?#2021-04-2916:09wilkerlucioI suggest you don’t do it this way, semantically resolvers should be free of side effects (exceptions are logging and caching), so better to break the process, read it first, and them use the returned value to side effect something. but your problem seems more related to caching, which I see as a different dimension, as I understand what you like to cache that external call in some DB, if that’s the case, you can write a custom cache store and them use it on the resolvers you want to cache (using https://pathom3.wsscode.com/docs/cache#custom-cache-store-per-resolver)#2021-04-2916:11wilkerlucioone difficulty when trying to save to a generic store is that the cache key of pathom uses rich data (vector with maps), which isn’t usually a good key for an external storage, the simple solution is use the hash of the cache-key, there is some collision risk here, so need to evaluate on a case by case basis#2021-04-2916:20wilkerlucioI have a system that I’m doing caching like that, but I decided to go with a simpler cache keying mechanism, this only stores strings (as I expect to read HTML, JSON, XML…)
(defrecord MamuteStringCacheStore [base-path]
  p.cache/CacheStore
  (-cache-lookup-or-miss
    [this cache-key f]
    (let [path (str base-path "/" cache-key)
          val  (and (fs/exists? path) (slurp path))]
      (if val
        val
        (let [val (f)]
          (if-not (string? val) (throw (ex-info "MamuteStringCache can only cache strings." {:value val})))
          (io/make-parents path)
          (spit path val)
          val)))))

(defn mamute-string-cache [base-path]
  (->MamuteStringCacheStore base-path))
#2021-04-2916:20wilkerlucioenv looks like:
(def env
  (-> {:wsscode/mamute-string (mamute/mamute-string-cache "/some/local/path")}
      (pci/register
        [...])
      (p.plugin/register (pbip/attribute-errors-plugin))))
#2021-04-2916:21wilkerlucioand a resolver getting geolocation data from Google Maps API:
(defn gcp-geolocation [address]
  (slurp
    (str
      ""
      "?address=" (url-encode address)
      "&key=" (get-key))))

(defn address-cache-key [address]
  (str "geocode/" (str/trim (str/replace address #"[^\w\s]+" "")) ".json"))

(pco/defresolver address-json [env {:keys [gcp.geo/address-string]}]
  {:gcp.geo/address-json
   (p.cache/cached :wsscode/mamute-string env (address-cache-key address-string)
     #(gcp-geolocation address-string))})
#2021-04-2916:21wilkerlucionote the usage of p.cache/cached inside of the resolver, this way I don’t have to deal with the resolver standard cache key on this case#2021-04-2916:37prncThe db would be datomic so maybe I could store ā€œrich dataā€, would have to look closer what it is. Thanks Wilker, will check out the docs on caching! Seems to be a good fit for data that has stable query -> result relationship ofc, maybe not for things that are changing e.g. ā€œget latest newsā€ etc. Cheers :beach_with_umbrella:#2021-04-2916:52prncBTW meta-data on the resolver output map will not survive the process, right? it’s replaced by pathom’s own meta not merge, if I understand correctly?#2021-04-2916:53prncI’m just thinking about some generic out-of band communication mechanism with the caller#2021-04-2916:59wilkerluciothe meta is preserved, but not from the resolver root return, its preserved from the process ā€œinitial dataā€, for example:
(meta (p.eql/process (pci/register [(pbir/constantly-resolver :x 10)])
     ^:with-meta {:foo "bar"}
     [:x]))
#2021-04-2916:59wilkerlucioin this case, you will see :with-meta in the output, because it was in the initial data#2021-04-2917:00wilkerlucioin case of resolvers that return maps (or sequences of maps), those maps are also initial data, and their meta should be preserved (and pathom adds the run stats on top of it)#2021-04-2917:00wilkerluciobut meta on the resolver responses (the root map) isn’t, because that is just a bag of keys getting merged in the entity#2021-04-2917:04prncI see, it seems I was thinking about that root map, I was just checking…#2021-04-2917:04prnc
(defresolver foobar [{foo :foo}]
    {::pco/output [:bar]}
    (with-meta
      {:bar {:bar-is :this-is-bar
                      :foo-is foo}}
      {:hello-meta "meta"}))

  (->> (p.eql/process (pci/register [foobar])
                  {:foo "hello"}
                  [:bar])
       meta
       :hello-meta)
  ;; => nil
#2021-04-2917:04prncThanks!#2021-05-0114:05markaddlemanUsing the latest Pathom Viz release and connector, I get the following stack trace every once in a while:
Sat May 01 08:03:46 MDT 2021 [worker-2] ERROR - POST /
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: No implementation of method: :-operation-config of protocol: #'com.wsscode.pathom3.connect.operation.protocols/IOperation found for class: com.wsscode.pathom3.connect.operation.Resolver
	at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
	at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2069)
	at clojure.core$deref_future.invokeStatic(core.clj:2304)
	at clojure.core$deref.invokeStatic(core.clj:2324)
	at clojure.core$deref.invoke(core.clj:2310)
	at com.wsscode.pathom.viz.ws_connector.impl.http_clj$handler.invokeStatic(http_clj.clj:39)
	at com.wsscode.pathom.viz.ws_connector.impl.http_clj$handler.invoke(http_clj.clj:30)
	at com.wsscode.pathom.viz.ws_connector.impl.http_clj$start_http_server_BANG_$fn__61908.invoke(http_clj.clj:67)
	at org.httpkit.server.HttpHandler.run(RingHandler.java:115)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at java.base/java.lang.Thread.run(Thread.java:831)
Caused by: java.lang.IllegalArgumentException: No implementation of method: :-operation-config of protocol: #'com.wsscode.pathom3.connect.operation.protocols/IOperation found for class: com.wsscode.pathom3.connect.operation.Resolver
	at clojure.core$_cache_protocol_fn.invokeStatic(core_deftype.clj:583)
	at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:575)
	at com.wsscode.pathom3.connect.operation.protocols$eval76622$fn__76634$G__76611__76639.invoke(protocols.cljc:3)
	at com.wsscode.pathom3.connect.operation$operation_config.invokeStatic(operation.cljc:112)
	at com.wsscode.pathom3.connect.operation$operation_config.invoke(operation.cljc:110)
	at com.wsscode.pathom3.connect.indexes$resolver_config.invokeStatic(indexes.cljc:213)
	at com.wsscode.pathom3.connect.indexes$resolver_config.invoke(indexes.cljc:207)
	at com.wsscode.pathom3.connect.indexes$resolver_optionals.invokeStatic(indexes.cljc:228)
	at com.wsscode.pathom3.connect.indexes$resolver_optionals.invoke(indexes.cljc:223)
	at com.wsscode.pathom3.connect.planner$resolvers_missing_optionals$fn__77815.invoke(planner.cljc:763)
	at clojure.core$map$fn__5880$fn__5881.invoke(core.clj:2746)
	at clojure.core.protocols$iter_reduce.invokeStatic(protocols.clj:49)
	at clojure.core.protocols$fn__8162.invokeStatic(protocols.clj:75)
	at clojure.core.protocols$fn__8162.invoke(protocols.clj:75)
	at clojure.core.protocols$fn__8110$G__8105__8123.invoke(protocols.clj:13)
	at clojure.core$transduce.invokeStatic(core.clj:6886)
	at clojure.core$transduce.invoke(core.clj:6872)
	at com.wsscode.pathom3.connect.planner$resolvers_missing_optionals.invokeStatic(planner.cljc:762)
	at com.wsscode.pathom3.connect.planner$resolvers_missing_optionals.invoke(planner.cljc:759)
	at com.wsscode.pathom3.connect.planner$compute_input_resolvers_graph.invokeStatic(planner.cljc:862)
	at com.wsscode.pathom3.connect.planner$compute_input_resolvers_graph.invoke(planner.cljc:844)
	at com.wsscode.pathom3.connect.planner$compute_attribute_graph_STAR_$fn__77870.invoke(planner.cljc:926)
	at clojure.lang.PersistentArrayMap.kvreduce(PersistentArrayMap.java:377)
	at clojure.core$fn__8460.invokeStatic(core.clj:6847)
	at clojure.core$fn__8460.invoke(core.clj:6832)
	at clojure.core.protocols$fn__8189$G__8184__8198.invoke(protocols.clj:175)
	at clojure.core$reduce_kv.invokeStatic(core.clj:6858)
	at clojure.core$reduce_kv.invoke(core.clj:6849)
	at com.wsscode.pathom3.connect.planner$compute_attribute_graph_STAR_.invokeStatic(planner.cljc:924)
	at com.wsscode.pathom3.connect.planner$compute_attribute_graph_STAR_.invoke(planner.cljc:892)
	at com.wsscode.pathom3.connect.planner$compute_attribute_graph.invokeStatic(planner.cljc:962)
	at com.wsscode.pathom3.connect.planner$compute_attribute_graph.invoke(planner.cljc:942)
	at com.wsscode.pathom3.connect.planner$compute_run_graph_STAR_$fn__77890.invoke(planner.cljc:1009)
	at clojure.lang.PersistentVector.reduce(PersistentVector.java:343)
	at clojure.core$reduce.invokeStatic(core.clj:6829)
	at clojure.core$reduce.invoke(core.clj:6812)
	at com.wsscode.pathom3.connect.planner$compute_run_graph_STAR_.invokeStatic(planner.cljc:997)
	at com.wsscode.pathom3.connect.planner$compute_run_graph_STAR_.invoke(planner.cljc:993)
	at com.wsscode.pathom3.connect.planner$compute_run_graph$fn__77901.invoke(planner.cljc:1090)
	at com.wsscode.pathom3.cache$eval77324$fn__77325.invoke(cache.cljc:19)
	at com.wsscode.pathom3.cache$eval77303$fn__77304$G__77294__77313.invoke(cache.cljc:10)
	at com.wsscode.pathom3.cache$cached.invokeStatic(cache.cljc:53)
	at com.wsscode.pathom3.cache$cached.invoke(cache.cljc:34)
	at com.wsscode.pathom3.connect.planner$compute_run_graph.invokeStatic(planner.cljc:1087)
	at com.wsscode.pathom3.connect.planner$compute_run_graph.invoke(planner.cljc:1029)
	at com.wsscode.pathom3.connect.planner$compute_run_graph.invokeStatic(planner.cljc:1073)
	at com.wsscode.pathom3.connect.planner$compute_run_graph.invoke(planner.cljc:1029)
	at com.wsscode.pathom3.connect.runner.async$plan_and_run_BANG_.invokeStatic(async.cljc:422)
	at com.wsscode.pathom3.connect.runner.async$plan_and_run_BANG_.invoke(async.cljc:413)
	at com.wsscode.pathom3.connect.runner.async$run_graph_impl_BANG_$fn__79531$fn__79532.invoke(async.cljc:505)
	at promesa.util.FunctionWrapper.apply(util.cljc:43)
	at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1183)
	at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2305)
	at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:143)
	at promesa.impl$eval56965$fn__56968.invoke(impl.cljc:154)
	at promesa.protocols$eval56417$fn__56544$G__56396__56557.invoke(protocols.cljc:28)
	at promesa.impl$eval56989$fn__56992.invoke(impl.cljc:246)
	at promesa.protocols$eval56417$fn__56544$G__56396__56557.invoke(protocols.cljc:28)
	at com.wsscode.pathom3.connect.runner.async$run_graph_impl_BANG_$fn__79531.invoke(async.cljc:504)
	at promesa.util.FunctionWrapper.apply(util.cljc:43)
	at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1183)
	at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2305)
	at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:143)
	at promesa.impl$eval56965$fn__56968.invoke(impl.cljc:154)
	at promesa.protocols$eval56417$fn__56544$G__56396__56557.invoke(protocols.cljc:28)
	at promesa.impl$eval57003$fn__57006.invoke(impl.cljc:266)
	at promesa.protocols$eval56417$fn__56544$G__56396__56557.invoke(protocols.cljc:28)
	at com.wsscode.pathom3.connect.runner.async$run_graph_impl_BANG_.invokeStatic(async.cljc:504)
	at com.wsscode.pathom3.connect.runner.async$run_graph_impl_BANG_.invoke(async.cljc:502)
	at com.wsscode.pathom3.plugin$run_with_plugins.invokeStatic(plugin.cljc:143)
	at com.wsscode.pathom3.plugin$run_with_plugins.invoke(plugin.cljc:129)
	at com.wsscode.pathom3.connect.runner$run_graph_with_plugins$fn__78242.invoke(runner.cljc:802)
	at com.wsscode.pathom3.plugin$run_with_plugins.invokeStatic(plugin.cljc:143)
	at com.wsscode.pathom3.plugin$run_with_plugins.invoke(plugin.cljc:129)
	at com.wsscode.pathom3.connect.runner$run_graph_with_plugins.invokeStatic(runner.cljc:800)
	at com.wsscode.pathom3.connect.runner$run_graph_with_plugins.invoke(runner.cljc:798)
	at com.wsscode.pathom3.connect.runner.async$run_graph_BANG_.invokeStatic(async.cljc:524)
	at com.wsscode.pathom3.connect.runner.async$run_graph_BANG_.invoke(async.cljc:517)
	at com.wsscode.pathom3.interface.async.eql$process_ast_STAR_$fn__57611$fn__57612.invoke(eql.cljc:16)
	at promesa.util.FunctionWrapper.apply(util.cljc:43)
	at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1183)
	at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2305)
	at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:143)
	at promesa.impl$eval56965$fn__56968.invoke(impl.cljc:154)
	at promesa.protocols$eval56417$fn__56544$G__56396__56557.invoke(protocols.cljc:28)
	at promesa.impl$eval56989$fn__56992.invoke(impl.cljc:246)
	at promesa.protocols$eval56417$fn__56544$G__56396__56557.invoke(protocols.cljc:28)
	at com.wsscode.pathom3.interface.async.eql$process_ast_STAR_$fn__57611.invoke(eql.cljc:15)
	at promesa.util.FunctionWrapper.apply(util.cljc:43)
	at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1183)
	at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2305)
	at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:143)
	at promesa.impl$eval56965$fn__56968.invoke(impl.cljc:154)
	at promesa.protocols$eval56417$fn__56544$G__56396__56557.invoke(protocols.cljc:28)
	at promesa.impl$eval57003$fn__57006.invoke(impl.cljc:266)
	at promesa.protocols$eval56417$fn__56544$G__56396__56557.invoke(protocols.cljc:28)
	at com.wsscode.pathom3.interface.async.eql$process_ast_STAR_.invokeStatic(eql.cljc:15)
	at com.wsscode.pathom3.interface.async.eql$process_ast_STAR_.invoke(eql.cljc:14)
	at com.wsscode.pathom3.plugin$run_with_plugins.invokeStatic(plugin.cljc:140)
	at com.wsscode.pathom3.plugin$run_with_plugins.invoke(plugin.cljc:129)
	at com.wsscode.pathom3.interface.async.eql$process_ast.invokeStatic(eql.cljc:23)
	at com.wsscode.pathom3.interface.async.eql$process_ast.invoke(eql.cljc:20)
	at com.wsscode.pathom3.interface.async.eql$process.invokeStatic(eql.cljc:51)
	at com.wsscode.pathom3.interface.async.eql$process.invoke(eql.cljc:26)
	at com.wsscode.pathom.viz.ws_connector.pathom3$connect_env$parser__62122.invoke(pathom3.cljc:224)
	at com.wsscode.pathom.viz.ws_connector.impl.http_clj$handler$fn__61895.invoke(http_clj.clj:41)
	at promesa.util.FunctionWrapper.apply(util.cljc:43)
	at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1183)
	at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2305)
	at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:143)
	at promesa.impl$eval56965$fn__56968.invoke(impl.cljc:154)
	at promesa.protocols$eval56417$fn__56544$G__56396__56557.invoke(protocols.cljc:28)
	at promesa.impl$eval57003$fn__57006.invoke(impl.cljc:266)
	at promesa.protocols$eval56417$fn__56544$G__56396__56557.invoke(protocols.cljc:28)
	at com.wsscode.pathom.viz.ws_connector.impl.http_clj$handler.invokeStatic(http_clj.clj:41)
	... 8 more
#2021-05-0114:54wilkerlucioin what context? what are you doing when the error happens?#2021-05-0115:47markaddlemanIt's seems to be random. I'll keep trying to find a pattern#2021-05-0117:05Bjƶrn Ebbinghaus@wilkerlucio I have a problem where I have a recursive query and I get nils in my response. (pathom 2) Query:
[{[:argument/id #uuid "608c8040-6722-4303-9801-e4f362e7fccb"]
  [:argument/id
   {:argument/premise->arguments 2}]}]
Response:
{[:argument/id #uuid "608c8040-6722-4303-9801-e4f362e7fccb"] 
 {:argument/id #uuid "608c8040-6722-4303-9801-e4f362e7fccb", 
  :argument/premise->arguments 
  [{:argument/id #uuid "608d46b9-3161-465c-9560-becbd20ff946", 
    :argument/premise->arguments 
    [{:argument/id #uuid "608d5014-012a-442f-ab72-885a9187ed8c", 
      :argument/premise->arguments 
      [nil nil]}]}]}                    <<----- These two nils are the problem
Actually my resolver (`{::pc/output [{:argument/premise->arguments [:argument/id]}]}` ) is called and returns two real values instead of the nils.
{:argument/premise->arguments
 [{:argument/id #uuid "608d7954-b05d-4f67-a44f-92b1c16984f4"}
  {:argument/id #uuid "608d7a34-7090-47c1-a855-47c8ee68baa0"}]}
I feel like the resolver shouldn't be called a third time? Do you know what the problem may be? It looks like there is a mismatch in the interpretation of the depth somewhere..?
#2021-05-0117:07Bjƶrn EbbinghausQuery:
[{[:argument/id #uuid "608c8040-6722-4303-9801-e4f362e7fccb"]
  [:argument/id
   {:argument/premise->arguments 0}]}]
Response:
{[:argument/id #uuid "608c8040-6722-4303-9801-e4f362e7fccb"] 
 {:argument/id #uuid "608c8040-6722-4303-9801-e4f362e7fccb"
  :argument/premise->arguments [nil nil nil nil]}}.  ; I removed three of the elements in the example above
#2021-05-0117:25Bjƶrn EbbinghausI see that https://github.com/wilkerlucio/pathom/blob/master/test/com/wsscode/pathom/core_test.cljc#L258 expects nil to be returned. The problem I face with the nil s is, that fulcro then tries to normalize them and then my db contains [:argument/id nil]s#2021-05-0401:38wilkerluciohi Bjorn, sorry the delay, the to-many case seems weird to have the items as nils#2021-05-0305:38henrik@wilkerlucio [Pathom 3] I just noticed that if I provide both an ident and params, (pco/params env) comes out as {}. Params do come through when I use a root query instead of an ident. Is this normal/intended/a known problem?#2021-05-0306:20nivekuiloh I ran into that too. forgot to follow up on it#2021-05-0307:09henrikI just worked around it like this for now:
(if (empty? params)
  (get-in env [::pcp/graph
               ::pcp/source-ast
               :params])
  params)
#2021-05-0312:40wilkerluciocan you send an example? without it its hard for me to understand what you are trying to do#2021-05-0400:45nivekuilis resolver-cache￱*￱ supposed to be derefed here? seem like the code shouldn't assume it's an atom but use the CacheStore protocol instead? https://github.com/wilkerlucio/pathom3/blob/21fad8734c7aa1f18cb02c0e2a72a5a6e1f38e91/src/main/com/wsscode/pathom3/connect/runner/async.cljc#L211#2021-05-0401:39wilkerluciogood catch, looks like a bug to me#2021-05-0402:41wilkerluciohttps://github.com/wilkerlucio/pathom3/issues/44#2021-05-0402:41wilkerlucioI expect to fix soon, but open the issue to keep track#2021-05-0404:06wilkerlucio@kevin842 fixed https://github.com/wilkerlucio/pathom3/pull/45#2021-05-0404:06wilkerlucio@kevin842 fixed https://github.com/wilkerlucio/pathom3/pull/45#2021-05-0414:04Björn EbbinghausDid someone here experiment with specing and gen-testing pathom mutations?#2021-05-0417:48royalaidWhile not directly I know @U2U78HT5G is playing around with Specing inputs to mutations, see his https://github.com/dehli/pathom3-plugins repo#2021-05-0509:59jmayaalv@wilkerlucio Not sure if an error on my side but when we run with guardrails enabled i get the following error
ERROR com/wsscode/pathom3/connect/runner.cljc:520 run-and-node!'s return type
....my huge env here...
should satisfy

  nil?
not pretty sure why, everything works fine if we disable guardrails, could it be something with the spec? this started to happen after the recent changes to the planner. The error starts to appear when running with this commit https://github.com/wilkerlucio/pathom3/commit/2863abe97d4c193d7ed5db31ad389fd5fded3197
#2021-05-0514:26wilkerluciothanks! the spec was wrong, fixed on master#2021-05-0521:31nivekuilpathom viz crash when I try to view a request, version 2021.4.22:
Listening to ipc renderer events
shared.js:166 Error: Cannot compare :date to 1
    at $APP.cljs.core.Keyword.cljs$core$IComparable$_compare$arity$2 (shared.js:4619)
    at Object.$APP.cljs.core._compare (shared.js:3103)
    at $APP.cljs.core.compare (shared.js:3344)
    at shared.js:4329
    at Object.$APP.cljs.core.tree_map_add (shared.js:4329)
    at $APP.cljs.core.PersistentTreeMap.cljs$core$IAssociative$_assoc$arity$3 (shared.js:4351)
    at $APP.cljs.core.PersistentTreeMap.cljs$core$ICollection$_conj$arity$2 (shared.js:4354)
    at $APP.cljs.core._conj (shared.js:3007)
    at Function.$APP.cljs.core.iter_reduce.cljs$core$IFn$_invoke$arity$3 (shared.js:3359)
    at Function.$APP.cljs.core.reduce.cljs$core$IFn$_invoke$arity$3 (shared.js:3364)
shared.js:8148 INFO [com.fulcrologic.fulcro.rendering.multiple-roots-renderer:152] - Optimized render failed. Falling back to root render.
shared.js:166 Error: Cannot compare :date to 1
    at $APP.cljs.core.Keyword.cljs$core$IComparable$_compare$arity$2 (shared.js:4619)
    at Object.$APP.cljs.core._compare (shared.js:3103)
    at $APP.cljs.core.compare (shared.js:3344)
    at shared.js:4329
    at Object.$APP.cljs.core.tree_map_add (shared.js:4329)
    at $APP.cljs.core.PersistentTreeMap.cljs$core$IAssociative$_assoc$arity$3 (shared.js:4351)
    at $APP.cljs.core.PersistentTreeMap.cljs$core$ICollection$_conj$arity$2 (shared.js:4354)
    at $APP.cljs.core._conj (shared.js:3007)
    at Function.$APP.cljs.core.iter_reduce.cljs$core$IFn$_invoke$arity$3 (shared.js:3359)
    at Function.$APP.cljs.core.reduce.cljs$core$IFn$_invoke$arity$3 (shared.js:3364)
shared.js:25 Uncaught Error: Cannot compare :date to 1
    at $APP.cljs.core.Keyword.cljs$core$IComparable$_compare$arity$2 (shared.js:4619)
    at Object.$APP.cljs.core._compare (shared.js:3103)
    at $APP.cljs.core.compare (shared.js:3344)
    at shared.js:4329
    at Object.$APP.cljs.core.tree_map_add (shared.js:4329)
    at $APP.cljs.core.PersistentTreeMap.cljs$core$IAssociative$_assoc$arity$3 (shared.js:4351)
    at $APP.cljs.core.PersistentTreeMap.cljs$core$ICollection$_conj$arity$2 (shared.js:4354)
    at $APP.cljs.core._conj (shared.js:3007)
    at Function.$APP.cljs.core.iter_reduce.cljs$core$IFn$_invoke$arity$3 (shared.js:3359)
    at Function.$APP.cljs.core.reduce.cljs$core$IFn$_invoke$arity$3 (shared.js:3364)
#2021-05-0521:32nivekuilworks on most requests, not sure what's different about that one. the tick tagged literal breaking it? do I need to add a transit handler somewhere#2021-05-0522:30nivekuilreturn value is [{0 0, :date #time/date "2021-05-05", 7 0, 20 0, 1 0, 4 0, 15 4.433987204485146, 21 0, 13 0, 22 0, 6 0, 17 0, 3 0, 12 0, 2 0, 23 0, 19 0, 11 0, 9 0, 5 0, 14 0, 16 0, 10 0, 18 0, 8 0}]#2021-05-0523:09wilkerluciook, I think I know what it is, pathom viz sorts the keys for readability#2021-05-0523:10wilkerlucioso its trying to compare the date key with the numbers, and that explodes#2021-05-0523:10wilkerluciocan you open an issue in pathom viz for that please?#2021-05-0600:08nivekuildone, thanks for looking into it#2021-05-0613:44Siddharth JainHi guys, In a query like this
[:catalogue/id
        {:products/search-results [:product/id :product/title :product/kind]}]
I want :products/search-results 's value sorted by :product/title , can I specify it here ?
#2021-05-0614:21Bjƶrn EbbinghausYou can just return the sorted list in your resolver. Alternatively, when you want your order to be dynamic, you can use query parameters: https://blog.wsscode.com/pathom/v2/pathom/2.2.0/connect/resolvers.html#_parameters#2021-05-0615:31Siddharth Jainthanks, I'll try šŸ‘#2021-05-0714:39Bjƶrn EbbinghausI opened a discussion on GitHub regarding authorization by an attribute (like id) across multiple resolvers. I would like for some thoughts on this: https://github.com/wilkerlucio/pathom/discussions/194 (I felt like my question was too long for slack and also slack forgets, while GitHub stays discoverable.)#2021-05-0715:07dehliIn case you haven’t seen @U2J4FRT2T’s repo yet, there’s some good discussion there as well. https://github.com/souenzzo/eql-style-guide/issues/4#2021-05-0716:31Tyler Nisonoff^ just responded to the post with a link to that! didn’t see the response here šŸ™‚#2021-05-1112:57Pragyan TripathiI am trying to use pathom-datomic while learning pathom and datomic. But I am getting following errors. Ideas on how can I resolve this?
Checking out:  at a1a48f96ea9df8461e4f473327b7de8b7e8f7734
Exception in thread "main" Syntax error compiling at (com/wsscode/pathom/connect/datomic.clj:1:1).
 at clojure.lang.Compiler.load(Compiler.java:7652)
 at clojure.lang.RT.loadResourceScript(RT.java:381)
 at clojure.lang.RT.loadResourceScript(RT.java:372)
 at clojure.lang.RT.load(RT.java:459)
 at clojure.lang.RT.load(RT.java:424)
 at clojure.core$load$fn__6856.invoke(core.clj:6115)
 at clojure.core$load.invokeStatic(core.clj:6114)
 at clojure.core$load.doInvoke(core.clj:6098)
 at clojure.lang.RestFn.invoke(RestFn.java:408)
 at clojure.core$load_one.invokeStatic(core.clj:5897)
 at clojure.core$load_one.invoke(core.clj:5892)
 at clojure.core$load_lib$fn__6796.invoke(core.clj:5937)
 at clojure.core$load_lib.invokeStatic(core.clj:5936)
 at clojure.core$load_lib.doInvoke(core.clj:5917)
 at clojure.lang.RestFn.applyTo(RestFn.java:142)
 at clojure.core$apply.invokeStatic(core.clj:669)
 at clojure.core$load_libs.invokeStatic(core.clj:5974)
 at clojure.core$load_libs.doInvoke(core.clj:5958)
 at clojure.lang.RestFn.applyTo(RestFn.java:137)
 at clojure.core$apply.invokeStatic(core.clj:669)
 at clojure.core$require.invokeStatic(core.clj:5996)
 at clojure.core$require.doInvoke(core.clj:5996)
 at clojure.lang.RestFn.invoke(RestFn.java:619)
 at com.vadelabs.studio.parser$eval150$loading__6737__auto____151.invoke(parser.clj:1)
 at com.vadelabs.studio.parser$eval150.invokeStatic(parser.clj:1)
 at com.vadelabs.studio.parser$eval150.invoke(parser.clj:1)
 at clojure.lang.Compiler.eval(Compiler.java:7181)
 at clojure.lang.Compiler.eval(Compiler.java:7170)
 at clojure.lang.Compiler.load(Compiler.java:7640)
 at clojure.lang.RT.loadResourceScript(RT.java:381)
 at clojure.lang.RT.loadResourceScript(RT.java:372)
 at clojure.lang.RT.load(RT.java:459)
 at clojure.lang.RT.load(RT.java:424)
 at clojure.core$load$fn__6856.invoke(core.clj:6115)
 at clojure.core$load.invokeStatic(core.clj:6114)
 at clojure.core$load.doInvoke(core.clj:6098)
 at clojure.lang.RestFn.invoke(RestFn.java:408)
 at clojure.core$load_one.invokeStatic(core.clj:5897)
 at clojure.core$load_one.invoke(core.clj:5892)
 at clojure.core$load_lib$fn__6796.invoke(core.clj:5937)
 at clojure.core$load_lib.invokeStatic(core.clj:5936)
 at clojure.core$load_lib.doInvoke(core.clj:5917)
 at clojure.lang.RestFn.applyTo(RestFn.java:142)
 at clojure.core$apply.invokeStatic(core.clj:669)
 at clojure.core$load_libs.invokeStatic(core.clj:5974)
 at clojure.core$load_libs.doInvoke(core.clj:5958)
 at clojure.lang.RestFn.applyTo(RestFn.java:137)
 at clojure.core$apply.invokeStatic(core.clj:669)
 at clojure.core$require.invokeStatic(core.clj:5996)
 at clojure.core$require.doInvoke(core.clj:5996)
 at clojure.lang.RestFn.invoke(RestFn.java:482)
 at com.vadelabs.studio.interface$eval144$loading__6737__auto____145.invoke(interface.clj:1)
 at com.vadelabs.studio.interface$eval144.invokeStatic(interface.clj:1)
 at com.vadelabs.studio.interface$eval144.invoke(interface.clj:1)
 at clojure.lang.Compiler.eval(Compiler.java:7181)
 at clojure.lang.Compiler.eval(Compiler.java:7170)
 at clojure.lang.Compiler.load(Compiler.java:7640)
 at clojure.lang.RT.loadResourceScript(RT.java:381)
 at clojure.lang.RT.loadResourceScript(RT.java:372)
 at clojure.lang.RT.load(RT.java:459)
 at clojure.lang.RT.load(RT.java:424)
 at clojure.core$load$fn__6856.invoke(core.clj:6115)
 at clojure.core$load.invokeStatic(core.clj:6114)
 at clojure.core$load.doInvoke(core.clj:6098)
 at clojure.lang.RestFn.invoke(RestFn.java:408)
 at clojure.core$load_one.invokeStatic(core.clj:5897)
 at clojure.core$load_one.invoke(core.clj:5892)
 at clojure.core$load_lib$fn__6796.invoke(core.clj:5937)
 at clojure.core$load_lib.invokeStatic(core.clj:5936)
 at clojure.core$load_lib.doInvoke(core.clj:5917)
 at clojure.lang.RestFn.applyTo(RestFn.java:142)
 at clojure.core$apply.invokeStatic(core.clj:669)
 at clojure.core$load_libs.invokeStatic(core.clj:5974)
 at clojure.core$load_libs.doInvoke(core.clj:5958)
 at clojure.lang.RestFn.applyTo(RestFn.java:137)
 at clojure.core$apply.invokeStatic(core.clj:669)
 at clojure.core$require.invokeStatic(core.clj:5996)
 at clojure.core$require.doInvoke(core.clj:5996)
 at clojure.lang.RestFn.invoke(RestFn.java:421)
 at user$eval138$loading__6737__auto____139.invoke(user.clj:1)
 at user$eval138.invokeStatic(user.clj:1)
 at user$eval138.invoke(user.clj:1)
 at clojure.lang.Compiler.eval(Compiler.java:7181)
 at clojure.lang.Compiler.eval(Compiler.java:7170)
 at clojure.lang.Compiler.load(Compiler.java:7640)
 at clojure.lang.RT.loadResourceScript(RT.java:381)
 at clojure.lang.RT.loadResourceScript(RT.java:368)
 at clojure.lang.RT.maybeLoadResourceScript(RT.java:364)
 at clojure.lang.RT.doInit(RT.java:486)
 at clojure.lang.RT.init(RT.java:467)
 at clojure.main.main(main.java:38)
Caused by: java.io.FileNotFoundException: Could not locate com/wsscode/pathom/connect/indexes__init.class, com/wsscode/pathom/connect/indexes.clj or com/wsscode/pathom/connect/indexes.cljc on classpath.
 at clojure.lang.RT.load(RT.java:462)
 at clojure.lang.RT.load(RT.java:424)
 at clojure.core$load$fn__6856.invoke(core.clj:6115)
 at clojure.core$load.invokeStatic(core.clj:6114)
 at clojure.core$load.doInvoke(core.clj:6098)
 at clojure.lang.RestFn.invoke(RestFn.java:408)
 at clojure.core$load_one.invokeStatic(core.clj:5897)
 at clojure.core$load_one.invoke(core.clj:5892)
 at clojure.core$load_lib$fn__6796.invoke(core.clj:5937)
 at clojure.core$load_lib.invokeStatic(core.clj:5936)
 at clojure.core$load_lib.doInvoke(core.clj:5917)
 at clojure.lang.RestFn.applyTo(RestFn.java:142)
 at clojure.core$apply.invokeStatic(core.clj:669)
 at clojure.core$load_libs.invokeStatic(core.clj:5974)
 at clojure.core$load_libs.doInvoke(core.clj:5958)
 at clojure.lang.RestFn.applyTo(RestFn.java:137)
 at clojure.core$apply.invokeStatic(core.clj:669)
 at clojure.core$require.invokeStatic(core.clj:5996)
 at clojure.core$require.doInvoke(core.clj:5996)
 at clojure.lang.RestFn.invoke(RestFn.java:619)
 at com.wsscode.pathom.connect.datomic$eval25742$loading__6737__auto____25743.invoke(datomic.clj:1)
 at com.wsscode.pathom.connect.datomic$eval25742.invokeStatic(datomic.clj:1)
 at com.wsscode.pathom.connect.datomic$eval25742.invoke(datomic.clj:1)
 at clojure.lang.Compiler.eval(Compiler.java:7181)
 at clojure.lang.Compiler.eval(Compiler.java:7170)
 at clojure.lang.Compiler.load(Compiler.java:7640)
#2021-05-1113:37tvaughanThis says Pathom isn't on your classpath#2021-05-1116:06Pragyan Tripathi:thinking_face: but it is…. my deps.edn has following:
com.wsscode/pathom     {:mvn/version "2.2.15"}
com.wsscode/pathom-datomic {:git/url ""
                                                      :sha     "a1a48f96ea9df8461e4f473327b7de8b7e8f7734 "}
#2021-05-1116:31tvaughanBoth of these are under :deps or the same alias?#2021-05-1116:33Pragyan Tripathiunder an alias - :dev#2021-05-1117:01wilkerluciocan you require pathom directly? also try killing .cpcache, sometimes it fixes deps issues#2021-05-1611:24Pragyan TripathiCleaning cache helped.. Now it works for my use case thanks a lot#2021-05-1201:37wilkerlucio#2021-05-1210:08jmayaalvHello @wilkerlucio i think we found a problem with batch processing but not sure if i am miss-understanding something on the way batch processing should work. i’ve created this gist https://gist.github.com/jmayaalv/b8cb9b8a6e465ef591773daab0829534 which hopefully can help explain the behavior we are experiencing. Basically the batch results get lost if there are a few batch resolvers involved in the query.#2021-05-1301:48wilkerluciohttps://github.com/wilkerlucio/pathom3/issues/49#2021-05-1301:49wilkerluciohttps://github.com/wilkerlucio/pathom3/pull/51#2021-05-1309:44jmayaalvi can confirm it works as expected now šŸ™‚ thanks a lot!#2021-05-1218:42royalaidHey all, is there a way to disable the log messages generated by Pathom Connect? Based on this https://github.com/wilkerlucio/pathom-viz-connector/blob/master/src/com/wsscode/pathom/viz/ws_connector/impl/sente_cljs.cljs#L80-L81 it doesn’t appear so?#2021-05-1218:50wilkerlucioI think not in general, but I'm open to change to log strategy that's more flexible#2021-05-1218:50wilkerlucioare there a lot of logs coming up that you want to get out?#2021-05-1220:24royalaidIt's moreso just a tad annoying that it happens whenever I reload the parser as my workflow is look at the console output and with nodes console logging the channels take up a bunch of space#2021-05-1220:25royalaidI will get an example up shortly#2021-05-1303:10royalaidApologies for the delay, here is what gets logged right now#2021-05-1303:10royalaidas you can see it goes on for a while so I just clear my console then execute the resolver#2021-05-1316:19wilkerlucio@U0S3YK6HK version 2021.05.13 of pathom-viz-connector now logs with Timbre, so you can configure it#2021-05-1322:34paul931224Hello everybody! I am using Pathom for a while now, it is fascinating, simplified my backend and frontend also. Now I try to do a file uploading example, but got myself in trouble.
(defmutation upload-files
  [env data]
  {::pco/op-name 'files/upload-files}
  (let [{:keys [directory-id files]} data]
    (println "Directory: " directory-id)
    "return"))

(p.eql/process
  env
  [`(files/upload-files {:directory-id directory-id})])
In the mutation the directory-id stays a symbol. I tried to (eval directory-id), but it didn't work. Also tried to expand my env with :
(def env
  (-> (pci/register registry)
      (p.plugin/register pbip/mutation-resolve-params)))
What should I do to make my params evaluate? I destructure them from the request.
#2021-05-1322:46souenzzo@paul931224 not a pathom issue, but an clojure-unquote issue. But file uploads may be a bit annoying to handle You are using fulcro? pedestal?
(p.eql/process
  env
  [`(files/upload-files ~{:directory-id directory-id})])
#2021-05-1322:52paul931224Thank you so much @souenzzo, I never had to use quote and eval in 6 years of clojure programming. Seems so easy now and trivial. We don't use nothing fancy like that, I get my files from multipart-params using ring . I made a route only for uploading the files, but wanted to do it with mutation . I have the files and everything, that was the last missing piece.#2021-05-1414:53wilkerlucio#2021-05-1415:40lilactowncan I set headers in pathom-viz?#2021-05-1416:06wilkerlucioin what context? adding a connection via HTTP with the + button?#2021-05-1416:11lilactownyeah, we have an API endpoint that uses pathom and I'd like to connect a remote pathom viz to it to e.g. show people how to write queries and basically show off pathom#2021-05-1416:12lilactownsimilar to what I did with GraphiQL at my last employer šŸ˜„ it made it much easier to onboard people and get them excited about it#2021-05-1416:12lilactownI haven't downloaded the latest release to check if its there so apologies for not doing my due diligence#2021-05-1416:34wilkerluciono worries#2021-05-1416:34wilkerluciothat connection part is being a bit neglected for some time, most because I personally tend to use the connector more, but I can see the connector is not a good option if you are looking to connect in some external service#2021-05-1416:38wilkerluciohttps://github.com/wilkerlucio/pathom-viz/issues/48#2021-05-1416:56lilactownmaybe i'll find some time to work on this#2021-05-1618:36cjsauerWith pathom 3's nested inputs, is there a limit to the level of nesting? I’m having an issue where inputs nested two levels deep are not working, but nested inputs at just one level deep work fine. There might be a different problem, but first I’m wondering if this limitation exists?#2021-05-1618:48cjsauerWas able to work around it with an intermediate resolver that ā€œbridges the gapā€, and I think this is actually cleaner than deeply nested inputs. So, if that limitation exists, I think it’s probably for good reason.#2021-05-1713:50wilkerlucioI don't wanna make a restriction on that, deep nested inputs should be supported, but at the same time, if you can break it down its better#2021-05-1707:56royalaidWould it be possible to have nice indentation of objects inside tagged values?#2021-05-1713:51wilkerlucioI use pprint to format those, I don't know if there is a way to format tagged values there#2021-05-1708:08royalaidAlso is there any way to manipulate or clear pathom-viz history?#2021-05-1713:55wilkerlucio#2021-05-1716:08royalaid:man-facepalming:#2021-05-1716:09royalaidThanks #2021-05-1721:47wilkerluciosorry, I though you were talking about request history, but from the git in your other PR I think you were talking about the query history, is that correct?#2021-05-1816:36royalaidYeah, that is correct. I just used the trash can icon but because that is fixed inside the history item you sometimes have to scroll horizontally inside the component to find it#2021-05-1817:41wilkerluciothe trash icon is a good point too, I think the icon should be fixed on the right side, no matter the scroll#2021-05-1817:43wilkerluciohttps://github.com/wilkerlucio/pathom-viz/issues/50#2021-05-1708:08royalaidIt seems I have hit some kind of limit and instead of new entries getting added it viz just overwrites the top most entry in the history#2021-05-1713:56wilkerlucioit should be removing from the tail instead, can you open an issue for that?#2021-05-1719:30royalaidhttps://github.com/wilkerlucio/pathom-viz/issues/49, I tried to capture a video for proof, let me know if anything else could be of help#2021-05-1721:45wilkerluciothanks!#2021-05-1903:15wilkerlucioI just merged a PR getting back some optimisations in Pathom 3! this is important to support the dynamic resolvers, which are also back in a small form, you can find more details about these changes at https://github.com/wilkerlucio/pathom3/pull/54#2021-05-1903:16wilkerlucioa more detailed description is coming on the next pathom updates blog post#2021-05-2314:56hadilsI am using Pathom 2.3.1 with Fulcro and I don't know why my mutations don't work with this parser:
(ns com.yardwerkz.local-store.parser
  (:require
    [com.wsscode.pathom.core :as p]
    [com.wsscode.pathom.connect :as pc]
    [cljs.core.async :as async :refer-macros [go]]
    [com.wsscode.async.async-cljs :refer [go-catch <!p <?]]
    [taoensso.timbre :as log]
    [com.wsscode.pathom.trace :as pt]
    [cognitect.transit :as t]
    ["@react-native-async-storage/async-storage" :default AsyncStorage]))

(defn set-item
  [key item]
  (let [w (t/writer :json)
        enc-item (t/write w item)]
    (.setItem AsyncStorage key enc-item)))

(defn get-item
  [key]
  (let [r (t/reader :json)]
    (.then (.getItem AsyncStorage key) #(t/read r %))))

(defn remove-item
  [key]
  (.then (.removeItem AsyncStorage key)))


;;;;; Resolvers for local-store must be written asynchronously.

(pc/defresolver index-explorer [env _]
  {::pc/input  #{:com.wsscode.pathom.viz.index-explorer/id}
   ::pc/output [:com.wsscode.pathom.viz.index-explorer/index]}
  {:com.wsscode.pathom.viz.index-explorer/index
   (get env ::pc/indexes)})

(pc/defmutation save-user-id [env {:keys [user-id]}]
  {::pc/input  #{:user-id}}
  (log/debug "save-user-id" user-id)
  (go
    (<!p (set-item "@user-id" user-id))))

(pc/defresolver get-user-id [_ _]
  {::pc/output [:user-id]}
  (go
    (let [user-id ("}}
                      ::p/mutate  pc/mutate-async
                      ::p/plugins [(pc/connect-plugin {::pc/register resolvers})
                                   p/error-handler-plugin
                                   p/trace-plugin]}))

(comment
  (go (prn (async/<! (parser {} ('save-user-id {:user-id "c"}))))))
When I put the line from the comment block into my CLJS repl, i get {}. If i put and undefined mutation symbol, I get {} . I would appreciate any help given. Thanks1
#2021-05-2316:14Chris O’DonnellYou need to put your quote outside the paren I think#2021-05-2316:14Chris O’DonnellRight now your symbol is trying to do a map lookup#2021-05-2317:27hadilsThank you @codonnell!#2021-05-2517:02fentonI'm trying to get pathom 3 to build with the latest rev but keep getting a null pointer exception when it tries to put together the classpath. The older (7 months ago) git sha:
{:paths ["src"]
 :deps
 {com.wsscode/pathom3
   {:git/url ""
    :sha "1f3ca76ead855609e0f27b30f6e8bf23b5bcfa0a"}}}
works and produces:
$ clj -X:deps tree 
org.clojure/clojure 1.10.3
  . org.clojure/spec.alpha 0.2.194
  . org.clojure/core.specs.alpha 0.2.56
com.wsscode/pathom3  1f3ca76
  . com.fulcrologic/guardrails 0.0.12
    . com.taoensso/timbre 4.10.0
      . com.taoensso/encore 2.91.0
        X org.clojure/tools.reader 0.10.0 :superseded
        . com.taoensso/truss 1.5.0
      . io.aviso/pretty 0.1.33
    . expound/expound 0.7.2
  . com.wsscode/async 1.0.8
    . org.clojure/core.async 1.1.587
      . org.clojure/tools.analyzer.jvm 1.0.0
        . org.clojure/tools.analyzer 1.0.0
        . org.clojure/core.memoize 0.8.2
          . org.clojure/core.cache 0.8.2
            . org.clojure/data.priority-map 0.0.7
        . org.ow2.asm/asm 5.2
        . org.clojure/tools.reader 1.3.2 :newer-version
  . com.wsscode/cljc-misc  66ba288
  . edn-query-language/eql 1.0.0
    X org.clojure/spec.alpha 0.2.176 :older-version
    X org.clojure/core.specs.alpha 0.2.44 :older-version
  . potemkin/potemkin 0.4.5
    . clj-tuple/clj-tuple 0.2.2
    . riddley/riddley 0.1.12
but the latest git sha:
{:paths ["src"]
     :deps
     {com.wsscode/pathom3
       {:git/url ""
        :sha "354574f6a4fbd30e54c85fa41d5bf2eb7de59a39"}}}
doesn't work, it produces:
$ clj -X:deps tree 
    Error generating tree: 
    java.lang.NullPointerException
       at clojure.tools.deps.alpha.util.dir$canonicalize.invokeStatic(dir.clj:30)
       at clojure.tools.deps.alpha.util.dir$canonicalize.invoke(dir.clj:25)
       at clojure.tools.deps.alpha.extensions.deps$eval1134$fn__1136.invoke(deps.clj:27)
       at clojure.lang.MultiFn.invoke(MultiFn.java:244)
       at clojure.tools.deps.alpha$expand_deps$children_task__534$fn__536$fn__537.invoke(alpha.clj:403)
       at clojure.tools.deps.alpha.util.concurrent$submit_task$task__249.invoke(concurrent.clj:34)
       at clojure.lang.AFn.call(AFn.java:18)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
When I'm inside pathom3 project, all is well, it's just when I include it as a dependency this happens. Any ideas?
#2021-05-2519:47wilkerluciohello @U0CGFT70T, I just tried to reproduce, but running on latest pathom sha I get the correct tree output#2021-05-2519:47wilkerlucioany chance some other dep may be conflicting? can you try in a bare project (just clojrue and pathom as deps) and see the error remains?#2021-05-2601:30fentonthx @U066U8JQJ I tried that...the deps.edn I show is the complete deps.edn file. I have other computers...I'll try on them and see if is somehow environmental?#2021-05-2517:03fentonWhen I try to clj-jack-in with cider I get the additional warnings: (which may just be red herrings of
error in process sentinel:
nrepl-server-sentinel: Could not start nREPL server:
DEPRECATED: Libs must be qualified,
change refactor-nrepl => refactor-nrepl/refactor-nrepl 
WARNING: Specified aliases are undeclared: [:dev]
Error building classpath. 
java.lang.NullPointerException
	at clojure.tools.deps.alpha.util.dir$canonicalize.invokeStatic(dir.clj:30)
	at clojure.tools.deps.alpha.util.dir$canonicalize.invoke(dir.clj:25)
	at clojure.tools.deps.alpha.extensions.deps$eval1390$fn__1392.invoke(deps.clj:27)
	at clojure.lang.MultiFn.invoke(MultiFn.java:244)
	at clojure.tools.deps.alpha$expand_deps$children_task__790$fn__792$fn__793.invoke(alpha.clj:403)
	at clojure.tools.deps.alpha.util.concurrent$submit_task$task__505.invoke(concurrent.clj:34)
	at clojure.lang.AFn.call(AFn.java:18)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
#2021-05-2518:37Jakub Holý (HolyJak)šŸ™ I discovered my understanding of Pathom 2 is lacking. I have a global resolver with ::pc/output {:teams [...]} and an empty input (and Pathom can see it, as per the screenshot) yet when I query for [:teams] I get back {:teams :com.wsscode.pathom.core/not-found} . What am I doing wrong? I expected the whole output of the resolver to be returned... .#2021-05-2518:40Jakub Holý (HolyJak)solved, the output lacked the [...] : ::pc/output [ {:teams [...]} ] .#2021-05-2608:40Tomas BrejlaI bumped into the same problem in the past as well. I wonder what's the expected contract here. Should you always provide all those keys "promised in pc/output" in output map and therefore return at least :teams [] in case no teams actually exist in your db? If so, perhaps some sort of key-presence validation (or clj-kondo rule?) could be added to avoid mistakes like the mentioned one?#2021-05-2518:53Jakub Holý (HolyJak)Another question: I am getting ::pc/reader-error with the message ":address/id is not ISeqable" when I query for a person's :address . The global resolver returns data containing: ... :address [:address/id 1] and I expect it to follow up with the other, ident resolver to get the actual data. Instead, I get the error above. Why?#2021-05-2518:53Jakub Holý (HolyJak)These are all my resolvers:
(defresolver my-very-awesome-teams [_ _] ; a global resolver
      {::pc/input  #{}
       ::pc/output [{:teams [:team/id :team/name
                             {:team/players [:player/id :player/name :player/address]}]}]}
      {:teams [#:team{:name "Hikers" :id :hikers
                      :players [#:player{:id 1 :name "Luna" :address [:address/id 1]}
                                #:player{:id 2 :name "Sol" :address [:address/id 2]}]}]})

    (defresolver address [_ {id :address/id :as in}] ; an ident resolver
      {::pc/input #{:address/id}
       ::pc/output [:address/id :address/city]}
      (println "RES address" in)
      (case id
        1 #:address{:id 1 :city "Oslo"}
        2 #:address{:id 2 :city "Trondheim"}))
#2021-05-2518:56Jakub Holý (HolyJak)The ident resolver works, as [{[:address/id 1] [:address/city]}] is correctly parsed into {[:address/id 1] {:address/city "Oslo"}} .#2021-05-2519:00Jakub Holý (HolyJak)If I change the data so that a player has a list of addresses instead of a single one (=> :address [[:address/id 1]]) then it works. But surely I should be able to have to-one references? What am I misunderstanding or doing wrong?#2021-05-2519:06Jakub Holý (HolyJak)Solved it, I was mixing up Fulcro ident's and Pathom's idea of identity. The correct value was :address {:address/id 1} , i.e. an entity with just the ID.#2021-05-2519:48wilkerluciojust seeing your messages now, glad you got it figured out šŸ™‚#2021-05-2520:59bmaddyI'm still new to Pathom. I'd like to use my resolvers to fill in the missing values of an entity I already have and wasn't able to figure it out from the docs. Something like this:
(def data
  {:first "Alyssa"
   :last "Hacker"
   :friends [{:first "Ben"
              :last "Bitdiddle"}
             {:first "Louis"
              :last "Reasoner"}]})
(parser {} [:first :full {:friends [:first :full]}])
=> {:first "Alyssa"
    :full "Alyssa Hacker"
    :friends [{:first "Ben"
               :full "Ben Bitdiddle"}
              {:first "Louis"
               :full "Louis Reasoner"}]}
I think I can skip the ident lookup by omitting pc/open-ident-reader, but how do I pass in the initial data? Also, a more general question, is this still a reasonable use of pathom?
#2021-05-2521:01wilkerluciohello, welcome šŸ™‚ to pass the data directly you can use:
(parser {::p/entity (atom DATA)} [:first :full ...]}
#2021-05-2521:11bmaddyIndeed--that works beautifully! Thank you!#2021-05-2813:34wilkerluciohello people, I've made a few changes to Pathom 3 error handling and add a documentation page about it in the docs, feedback welcome as alwaysĀ šŸ™Ā Ā https://pathom3.wsscode.com/docs/error-handling#2021-05-3120:04tekacsYou don’t seem to mention in the docs what happens if multiple mutations are run and there’s an error in one — whether it continues executing the remaining mutations or stops. Just might be worth mentioning for newcomers who might expect one behaviour or the other?#2021-05-2814:27wilkerlucioAlso extended docs around resolver prioritization https://pathom3.wsscode.com/docs/resolvers/#prioritization to cover edge cases and point out this still experimental#2021-05-2817:57henrikBtw, @wilkerlucio, I managed to blow my heap by making a big query the other day. Just an FYI, I've broken it up into smaller queries.#2021-05-2817:58wilkerluciodo you know if exploded during plan or run?#2021-05-2817:59henrikI think it was during run. If it's interesting, I could check out the old code and try to figure out which it is.#2021-05-2818:00wilkerlucioI'm interested, the goal is to support arbitrarily large queries, good to know where the current implementation is topping#2021-05-2818:01wilkerlucioyour guess would be because of query size, data size (large number of entities) or both?#2021-05-2818:03wilkerlucioalso curious about the available resources on the machine, does increasing the max size is an option for you to fix?#2021-05-2820:05wilkerluciosorry the question storm, but one other thing to check, are you running with guardrails enabled?#2021-05-3100:13wilkerlucio#2021-05-3107:18nivekuilso with distributed resolvers, you can do something like build a read-through cache with client-side pathom resolvers?#2021-05-3112:46wilkerluciowhat you mean by read-though cache in this context?#2021-05-3113:14nivekuillike, fetch an attribute from local cache if it's there, otherwise transparently load it from remote#2021-05-3113:16nivekuiland then cache that remote result in the local cache#2021-05-3114:15wilkerlucioin theory, yes, but its not so easy, caching of dynamic resolvers is a hard problem given their complex inputs/output patterns#2021-06-0115:02jmayaalvwe are getting some unexpected errors with the latest changes to error handling. ex:
[{[:edge.account/code "account-1"]
  [{:edge.account/contracts
    [:edge.contract/code :edge.money/currency]}]}]
=>
{[:edge.account/code "account-1"]
 {:com.wsscode.pathom3.connect.runner/attribute-errors
  {:edge.account/contracts
   {:com.wsscode.pathom3.error/error-type
    :com.wsscode.pathom3.error/node-errors,
    :com.wsscode.pathom3.error/node-error-details {}}},
  :edge.account/contracts
  [{:edge.contract/code "contract-1",
    :edge.money/currency
    #object[Object [TaggedValue: unknown, #currency "USD"]]}]}}
what am i missing? the attrs were found, so why the errors?
#2021-06-0115:33wilkerluciothis might be a bug, I can look into that in a few hours#2021-06-0115:33wilkerluciocan you open an issue on pathom 3 repo please?#2021-06-0116:05jmayaalvsure, will do#2021-06-0223:28wilkerluciohttps://github.com/wilkerlucio/pathom3/issues/59 fixed on master#2021-06-0306:44jmayaalvThanks a lot @U066U8JQJ tried and working and expected. we don't use placeholders much yet but I'll check if we can run some tests.#2021-06-0223:28wilkerluciohttps://github.com/wilkerlucio/pathom3/issues/59 fixed on master#2021-06-0218:26markaddleman:b attribute is unreachable but I think it should work#2021-06-0218:28wilkerlucioThe problem here is that you trying to override the value of :doc, Pathom doesn't do that#2021-06-0218:28markaddlemanah, I see. That would explain some other wierdness#2021-06-0218:29wilkerlucioPathom can go down and further process an entity, but from the parent level you can only set the value for that property once#2021-06-0218:51wilkerlucioone thing to help thinking though this, when processing the attributes for an entity, the entity starts with the initial available data, and merges new data as it calls resolvers, during merge, it will never override a value that's already there, once a value is set on a entity, that value will never be replaced (or merged, since Pathom sees the value in the entity, pathom considers it done at that level)#2021-06-0218:56markaddleman^^ This is the key#2021-06-0218:56markaddlemanThaanks!#2021-06-0404:04bmaddyI suspect I'm missing something silly here, but how do I do the initial entity lookup? My query looks like this:
[{[:user/id #uuid "5f3bf49f-36fa-41a8-88dd-09ca67f8392f"]
  [:db/id]}]
I'm pretty sure I want a resolver (not a reader), but not positive. I expected it to convert [:user/id #uuid "5f3bf49f-36fa-41a8-88dd-09ca67f8392f"] to {:user/id #uuid "5f3bf49f-36fa-41a8-88dd-09ca67f8392f"} internally and trigger my {::pc/input #{:user/id} ::pc/output [:db/id]} resolver, but it never runs. Is there a simple example of this somewhere? I can't find it in the docs.
#2021-06-0408:22Bjƶrn EbbinghausYou have to use an ident reader for pathom to understand idents. E.g. pc/open-ident-reader#2021-06-0413:21wilkerluciocan you please share the parser setup? the query looks correct, maybe there is some setup missing#2021-06-0413:36bmaddyYeah, here's the whole thing.#2021-06-0413:41bmaddyI should mention, this is pathom 2.2.31.#2021-06-0413:45wilkerlucioone minor error in the example code is that you define p and try to call parser#2021-06-0413:45wilkerluciothis is working here (minor changes to your code, so I don't depend on datascript/datomic):
(pc/defresolver user-resolver
  [{:keys [db]} {:keys [user/id]}]
  {::pc/input  #{:user/id}
   ::pc/output [:db/id]}
  (println :user-resolver) ;; never prints
  {:db/id (get-in db [:user/id id])})

(def p
  (p/parser
    {::p/env     {::p/reader [p/map-reader
                              pc/reader2
                              pc/open-ident-reader
                              p/env-placeholder-reader]}
     ::p/plugins [(pc/connect-plugin {::pc/register [user-resolver]})
                  p/error-handler-plugin]}))
(comment
  (let [db {}]
    (p {:db db}
      [{[:user/id #uuid "5f3bf49f-36fa-41a8-88dd-09ca67f8392f"]
        [:db/id]}]))
  )
#2021-06-0413:45wilkerlucio
(let [db {}]
    (p {:db db}
      [{[:user/id #uuid "5f3bf49f-36fa-41a8-88dd-09ca67f8392f"]
        [:db/id]}]))
:user-resolver
=> {[:user/id #uuid"5f3bf49f-36fa-41a8-88dd-09ca67f8392f"] #:db{:id nil}}
#2021-06-0413:46bmaddyOh geez. facepalm#2021-06-0413:48bmaddyI guess I was missing something silly. Thanks for taking a look.#2021-06-0413:48wilkerluciono worries, glad to help#2021-06-0521:40mauricio.szaboHi there! @wilkerlucio I was talking with you in private about some resolver problems, and here's what I found: On my current Duck-REPLed project, I'm able to query REPL interactions. To make this happen, I somehow need to know what to query. To make things as simple as possible, I decided to do some "default contents", for example: if I do query [:var/fqn] I probably want the "full qualified name" of the current var in the editor.#2021-06-0521:41mauricio.szaboThis made the planner incredibly slow - it's taking +500ms to plan the query. How can I send you more info, so you can check if it's some bug or it's inherently a problem on how I made my resolvers?#2021-06-0521:44mauricio.szaboMore info about this problem: here's the "slow" query:#2021-06-0521:44mauricio.szaboAnd here's the fast one:#2021-06-0523:41wilkerluciohello, can you send an example of how I can reproduce it?#2021-06-0615:19mauricio.szaboHelpers is duck-repled.repl-helpers eql is generated with duck-repled.core/gen-eql#2021-06-0615:19mauricio.szaboOk, in theory if you start a REPL with duck-repled and try to evaluate these codes, it should have the same problem that I had#2021-06-0615:20mauricio.szaboThe problem is that I'm not being able to connect Viz on them: on Clojure, it always return nil for the eval result, and not give me the trace, and in ClojureScript it's crashing node with:
INFO [com.wsscode.pathom.viz.ws-connector.impl.sente-cljs:81] - Connecting to websocket {:com.wsscode.pathom.viz.ws-connector.core/on-message #object[Function], :com.wsscode.pathom.viz.ws-connector.impl.sente-cljs/send-ch #object[cljs.core.async.impl.channels.ManyToManyChannel], :com.wsscode.pathom.viz.ws-connector.core/parser-id :duck-repled.core/duck}
INFO [com.wsscode.pathom.viz.ws-connector.impl.sente-cljs:62] - Waiting for channel to be ready 1000
INFO [com.wsscode.pathom.viz.ws-connector.impl.sente-cljs:75] - Waiting for channel to be ready 1000

/home/mauricio/projects/duck-repled/.shadow-cljs/builds/tests/dev/out/cljs-runtime/cognitect/transit.cljs:418
        (WithMeta. (-with-meta ^not-native x nil) m)
        ^
TypeError: x.cljs$core$IWithMeta$_with_meta$arity$2 is not a function
    at Transit$JSONMarshaller.transform (/home/mauricio/projects/duck-repled/.shadow-cljs/builds/tests/dev/out/cljs-runtime/cognitect/transit.cljs:418:9)
#2021-06-0615:21mauricio.szaboI was only be able to truly connect to Viz and check the query inside the Chlorine project šŸ˜ž But then, it's quite hard to setup everything to make it work#2021-06-0815:00MatthewLispHi everyone! I'm taking a look at the docs/guide regarding Pathom plugins, and i'm seeing this snippet
; define the extension wrapper fn
(defn protect-attributes-wrapper [mse]
 (fn [env source {:keys [key] :as ast}]
   (if (and (contains? source key)
            (contains? protected-attributes key))
 ; the output of this extension must be a map entry or nil
 ; a vector with two elements would also work, but creating a map entry is
 ; more efficient
 (coll/make-map-entry key ::protected-value)
 (mse env source ast))))
I'm wondering about the comment that states the output of this extension must be a map entry or nil However i see that the output is actually a call to the original function, in this example called mse What i'm missing here?
#2021-06-0815:08jmayaalvHi @matthewlisp, the call to (mse env source ast) is in the else leg of the condition.#2021-06-0815:08jmayaalvso in that sample (coll/make-map-entry key ::protected-value) it’s called if something needs to be hidden, if not continue with the normal execution: (mse env source ast)#2021-06-0815:12MatthewLispwow, right, the comment lines made me forget that we were inside an if statement haha, thanks#2021-06-0817:52bmaddyDoes anyone know where I could find a simple example of how to connect pathom to datomic (or any other database)? I've tried doing it in a resolver, but it doesn't seem to recurse into child entities so my other resolvers don't get triggered.
(pc/defresolver bill-id-resolver
  [{:keys [db]} {:keys [bill/id]}]
  {::pc/input #{:bill/id}
   ::pc/output all-datomic-attributes}
  ;; {:db/id (:db/id (d/entity db [:bill/id id]))}
  ;; (select-keys (d/entity db [:bill/id id]) all-datomic-attributes)
  (d/pull db all-datomic-attributes [:bill/id id]))
Then I tried in a reader, but it's only getting triggered for the initial lookup-ref and not attributes in the associated pull expression.
(defn datomic-reader
  [{:keys [db ast] :as env}]
  (let [lookup-ref ((juxt p/ident-key p/ident-value) env)
        k (:dispatch-key ast)
        e (d/entity db lookup-ref)
        v (get e k)]
    (println lookup-ref k e v)
    (if v
      {k v}
      ::p/continue)))
Here's the parser I'm using for the reader version. The last-update-t-resolver is just a simple resolver that makes derived data. It works just fine in other parsers.
(def registry
  "All the resolvers to register with the parser."
  [#_bill-id-resolver
   last-update-t-resolver])

(def parser
  (p/parser
   {::p/env     {::p/reader [;; pass through values from the original entity map
                             p/map-reader
                             ;; look up most values in datomic
                             datomic-reader
                             ;; use resolvers for data that doesn't exist yet
                             pc/reader2
                             pc/open-ident-reader
                             ]}
    ::p/plugins [(pc/connect-plugin {::pc/register registry})
                 p/error-handler-plugin]}))
#2021-06-0915:13wilkerlucioreader is a low level abstraction, the resolvers are the correct layer to do that#2021-06-0822:36bmaddyOops, forgot to include how I'm calling it:
(parser {:db (d/db conn)}
          [{[:ti.bill/id #uuid "799ce7c1-cade-4ecc-8a89-a1df186d724e"]
            [:db/id
             :ti.bill/pro]}])
#2021-06-0914:17markaddlemanI have a resolver that takes a few required attributes and an optional attribute. In a particular query, the optional attribute cannot be computed by its resolver. My environment contains the attribute-errors-plugin. I see that the attribute-errors-plugin reports an "Insufficient data" for the optional node/attribute.#2021-06-0914:19markaddlemanA few questions: Is this intended behavior? If so, I'd like to understand the rationale. If this is not intended behavior, I'll produce a minimal test case#2021-06-0914:20markaddlemanOh, one more thing: The optional attribute is declared within pco/input as
{(pco/? :data-source.event/attributes) [:data-source.event.attribute/id
                                                           :data-source.event.attribute/friendlyName
                                                           :ui.attribute.selector/heading]}
I think that's the right syntax
#2021-06-0915:13wilkerlucio@markaddleman can you send a full example? may be a bug#2021-06-0915:15markaddlemanI can easily send a the exception that attribute-errors plugin reports. Creating a minimal test case is possible, of course, but it will take me a day or two#2021-06-0915:18wilkerluciojust a minimal repro so I can see what you see#2021-06-1714:36markaddlemanHi. I'm finally picking this back up and it turns out the repro case is very simple:
(pco/defresolver data-source-event-attributes []
  {::pco/input  []
   ::pco/output [:out]})

(def env (-> (pci/register [data-source-event-attributes])
             (p.plugin/register (pbip/attribute-errors-plugin))))

(comment
  (p.eql/process env [{'(:>/GroupBys {}) [(pco/? :out)]}]))
The query produces an attribute error. I don't think it should
#2021-06-1715:49wilkerlucioquerying optional items in eql/process isnt a thing#2021-06-1715:49wilkerluciothey only work in resolvers, not in eql process#2021-06-1716:36markaddlemanOh šŸ˜• I have a resolver that does this query on its input. I figured I was simplifying by moving it to the client query#2021-06-1716:37markaddlemanI'll keep trying to find a small repro case#2021-06-1719:16wilkerlucioand just to clear it up, they are not a thing there because there is no dependency on top of that, so if its not there, it just an edge thing, but maybe it could have some value in the sense of validation, not a thing at this moment, but if you believe this is a useful case please open a discussion on pathom 3 repo and we can keep that in mind#2021-06-1719:19markaddlemanI just got a minimal case. I'll open a discussion on github#2021-06-1719:21markaddlemanactually, my repro case may have found something different. I have a case where pathom cannot generate a plan where I think it should#2021-06-1719:21markaddlemanPutting into github#2021-06-1719:23markaddlemanhttps://github.com/wilkerlucio/pathom3/discussions/61#2021-06-1811:48wilkerlucioFixed on main#2021-06-1812:51markaddlemanThanks!#2021-06-1811:48wilkerlucioFixed on main#2021-06-1015:11markaddlemanHi @wilkerlucio - I have a plugin that I want to run at the very end of processing, just before results are returned to the client. In particular, I want the results to be sorted for the UI. I thinik the closest fit to needs is
:com.wsscode.pathom3.format.eql/wrap-map-select-entry
but I'm not sure how the plugin should detect that processing is at the end.
#2021-06-1015:56wilkerluciomap-select-entry is when pathom merges each attribute at the entity#2021-06-1015:56wilkerluciofor all around, you can use https://pathom3.wsscode.com/docs/plugins#pcrwrap-root-run-graph or https://pathom3.wsscode.com/docs/plugins#peqlwrap-process-ast#2021-06-1015:57wilkerluciohttps://pathom3.wsscode.com/docs/plugins#pcrwrap-run-graph is per entity#2021-06-1017:56markaddlemanThanks!#2021-06-1301:10nivekuilin theory is it possible to have pathom do a join without explicitly returning a map in the resolver? e.g. instead of returning {:join-from {:join-to 1}} you just return {:join-from 1} and specify externally that the attribute :join-from can be walked to reach :join-to, so the query [{:join-from [:join-to]}] still works?#2021-06-1302:44wilkerlucionot by default, you can make something like that via plugins, but that may lead to some confusion on the users (value changing types depending on query shape)#2021-06-1316:10wilkerlucioHello everyone! Today I have some news on Pathom docs! 1. Now for async processes, the env can be a promise, this allows the user to make an easy async build-up, for loading external indexes for example. https://pathom3.wsscode.com/docs/async#async-env 2. Boundary interface just got it's section on the documentation. https://pathom3.wsscode.com/docs/eql#boundary-interface 3. A new tutorial is out! Learn how to deploy Pathom as a Google Cloud Function: https://pathom3.wsscode.com/docs/tutorials/serverless-pathom-gcf Another important one, if you are already using the boundary interface, I'm renaming :pathom/tx to :pathom/eql, if you used that please upgrade in your code (`:pathom/tx` is still supported for now, but I'll remove it once Pathom 3 reaches beta).#2021-06-1511:20fatihictI have a question about error handling in Pathom. I'm using Pathom with the sha 304a6b3a56d588eab50f7f6eaf2c2c7b31750782 The code below returns the attribute errors as I would expect, but the same code returns an empty map when using the latest sha acf3134aa52668d83cffd326c46fca1ef0120e99.
(p.eql/process
    (-> (pci/register
          (pbir/constantly-fn-resolver :error-demo
            (fn [_] (throw (ex-info "Example Error" {})))))
      (p.plugin/register
        [pbip/remove-stats-plugin
         (pbip/attribute-errors-plugin)]))
    [:error-demo])
=> {:com.wsscode.pathom3.connect.runner/attribute-errors {:error-demo #error{:message "Example Error", :data {}}}} 
#2021-06-1513:24wilkerluciohello @U28947274, great to see you around again šŸ™‚#2021-06-1513:24wilkerlucioI checked it out, and indeed it as a problem, you can fix it by removing the pbip/remove-stats-plugin#2021-06-1513:25wilkerlucioI'm thinking about some of the plugin architecture, I noticed later in the game a problem relateed to the idea I came up when combined with batch resolvers#2021-06-1513:26wilkerluciothat lead to a need to change the implementation of attribute-errors-plugin, and now I see the new way isn't compatible with remove-stats-plugin anymore#2021-06-1513:27fatihictHey Wilker šŸ‘‹#2021-06-1513:27fatihictAh oke, that is good to know. Thanks for clarifying.#2021-06-1520:51mauricio.szaboHi, @wilkerlucio. I've been trying to use priority on the resolvers on Pathom3, and I have to be honest: I keep getting unpredictable results. I'll try to hack something to see if I can somewhat change the prioritization, and I know we already discussed this on https://github.com/wilkerlucio/pathom3/discussions/57 - I'm just giving some feedback on my feelings about this new feature šŸ™‚#2021-06-1520:52wilkerluciothanks, as we talked before, this still a first idea for it, glad that you are finding issues, can you add more details on the discussion? explain some case that doesn't work, so we can think about how we can make it better#2021-06-1520:55mauricio.szaboRight, so the case I sent on the discussion was one example. What I find is that weights more in the prioritization is mostly resolved on intermediate steps to get to the goal. So sometimes you ask for :a, and :a is reachable both from a "2" and "1" priority, but what weights more are intermediate steps that may not exist on both graphs on the path to get :a#2021-06-1520:56mauricio.szabo(I don't know if my last explanation made any sense šŸ˜„)#2021-06-1521:14mauricio.szaboAlso, another question: is there a way (planned or not) to get, from a resolver, which attributes were asked on the query? I'm thinking about a resolver, for example, that outputs [:person/name :person/age :person/id] and can somehow issue a SQL query with only the attributes that the query asked...#2021-06-1521:28wilkerluciothis is a problem for dynamic resolvers, which is the thing I'm currently working on#2021-06-1522:17mauricio.szaboIs there any documentation on how they will work in the future, or something? šŸ™‚#2021-06-1523:02wilkerlucionot yet, but maybe you can work some partial thing with the current stuff#2021-06-1523:03wilkerlucioto answer your question more directly, inside env, if you get the ::pcp/node, inside of it there is ::pcp/expects, which is the attributes expected from the resolver#2021-06-1523:45mauricio.szaboNice, will try it soon-ish šŸ™‚#2021-06-1523:45mauricio.szaboAlso, one last question: When something fails, it shows some errors like:
:definition/contents
   {:com.wsscode.pathom3.error/error-type
    :com.wsscode.pathom3.error/node-errors,
    :com.wsscode.pathom3.error/node-error-details
    {3
     {:com.wsscode.pathom3.error/error-type
      :com.wsscode.pathom3.error/attribute-missing},
     4
     {:com.wsscode.pathom3.error/error-type
      :com.wsscode.pathom3.error/attribute-missing}}}},
Is it possible to also add the resolver name, so it's easier to debug problems? (also, great work on this feature, it helps a lot!)
#2021-06-1601:41stuartrexkingPathom 2.2.0. How do I query for a vector of vectors for a global resolver?
(pc/defresolver connections [{::datomic/keys [conn] :as env} _]
  {::pc/output [{::connection/connections [::connection/name ::connection/id]}]}
  (let [{:keys [context rel]} (-> env :ast :params)]
    {::connection/connections
     (cc/path (d/db conn) context rel)}))
#2021-06-1601:42stuartrexkingThe output is a vector of vectors {::connection/connections [[…]]}#2021-06-1601:42stuartrexkingIs it possible to do such a thing?#2021-06-1602:20wilkerlucio@stuartrexking no support for vectors of vectors, you can instead make a vector of maps, on where each map has an attribute that is another vector#2021-06-1721:56lsenjovPathom 2.3.1. Is there a way to return nil in a field in a resolver instead of ::p/not-found? My google-fu is failing me#2021-06-1721:59wilkerluciothere is a plugin that can help you there:
(def parser
  (p/parser {::p/plugins [;; add this one at the end of your plugins
                          p/elide-special-outputs-plugin]}))
#2021-06-1722:00wilkerlucioand nothing wrong with your google-fu, just not well documented#2021-06-1722:01lsenjovI’m not looking to replace every instance of not found, just the ones I know are nil. Would writing a plugin that replaces something like ::p/nil with nil on output be a reasonable solution?#2021-06-1722:02wilkerlucioif you look into the p/elide-special-outputs-plugin, you can copy and modify it#2021-06-1722:03wilkerlucio
(def elide-special-outputs-plugin
  (post-process-parser-plugin elide-special-outputs))
#2021-06-1722:03wilkerlucio
(defn elide-special-outputs
  "Convert all ::p/not-found values of maps to nil"
  [input]
  (elide-items special-outputs input))
#2021-06-1722:03wilkerlucio
(def special-outputs #{::reader-error ::not-found})
#2021-06-1722:04wilkerlucioso, something like: (p/post-process-parser-plugin #(elide-items #{::p/not-found} %))#2021-06-1722:07lsenjovOh I see! Thank you for the help šŸ™‚#2021-06-1811:30wilkerluciooh, I just see there is p/elide-not-found that does it already#2021-06-2312:25henrikI'm having trouble getting Pathom(3) to make use of batch resolvers. I seem to be fulfilling the criteria for batch resolvers to kick in (maps are returned, or rather, a vector of maps), but Pathom seems to split them into individual calls anyway. I'm not sure what to look for while investigating this issue, what should I be looking out for?#2021-06-2312:31jmayaalvcan you share some code? we use batch resolvers on pathom3 and havent’ had such problem.#2021-06-2312:43henrik#2021-06-2314:23wilkerluciohello Henrik, looking at the graph it seems to be using batch correctly, that blue bar is kind of a ā€œshared timeā€ in the batch of the items#2021-06-2314:24wilkerluciohave you tried logging to see if there are many calls to the resolver?#2021-06-2315:43wilkerluciomaybe the reading is a bit confusing, but see that those blue bars are in "parallel", altough there is no parallel runner yet, the intention was to make an indication they are a single thing (the batch run), do you have an idea how we could make this clear?#2021-06-2316:46henrikAha! That's interesting. I've also written a plugin to wrap each resolver call with a log to Honeycomb, where it looks like it's registering numerous calls to the same resolver:#2021-06-2316:48henrikNow I'm thinking that I might not be measuring what I think I'm measuring. When wrapping around ::pcr/wrap-resolve, am I wrapping around the call to the resolver, or the construction of the resolver, or something else?#2021-06-2316:50henrik(Btw, Honeycomb might be an example of a UX that is quite good at communicating where spans are children vs. siblings)#2021-06-2316:54henrik
(defn wrap-resolver-with-honeycomb [resolve]
  (fn [env node]
    (wrap-honeycomb env (str (::pco/op-name node))
      (resolve env node))))
#2021-06-2320:17henrikSo, It's pretty clear to me that I've misunderstood wrap-resolve, and that this is the source of my confusion. Is there a way to wrap the invocation of a resolver, at the moment where it receives the inputs?
#2021-06-2322:44wilkerluciothis expectation is correct, but as you are seeing, in the case of batch it get some caveats#2021-06-2322:44wilkerlucioindeed it always runs when the resolver gets the input, but, in the case of batch, it doesn't run immediatly#2021-06-2322:45wilkerlucioyou can check if that happen, by looking at the output of what you are wrapping, in case of batch it will be a map with the key ::pcr/batch-hold on it, if you see that, this means the resolver isn't running now, and will run later as a batch#2021-06-2322:45wilkerluciofor observability purposes, you may want to ignore those cases#2021-06-2322:48wilkerluciohoneycomb looks nice šŸ™‚#2021-06-2405:20henrik> for observability purposes, you may want to ignore those cases Right, but I also need some way of running it when the batch resolver finally runs. Is this possible? > honeycomb looks nice It is! Highly recommended.#2021-06-2405:28henrikOh dear, and the result is a map with ::pcr/batch-hold on it. By then it is far too late to decide not to run Honeycomb. Honeycomb must be running while the resolver is evaluating… Do you see? It can't wait until after the resolver has run to evaluate the performance. In that case I'd have to run each resolver twice: first once, to detect whether it is a batch, then again to actually measure it.#2021-06-2405:33henrikI need either a way to determine ahead of time that it is a batch resolver, AND a way to run a plugin when the batch is actually executed, or (and this would be the vastly preferred case) I need an extension point that works the same way for batching and non-batching resolvers: i.e., it that runs on the actual execution of the resolvers, regardless of kind, and not on any preparatory step.#2021-06-2405:40henrikThe first case would make it work, but it would also required two separate plugins to handle the one use case. The second case would make it work, and only require the one extension point.#2021-06-2415:02wilkerluciogood to hear about this case#2021-06-2417:01wilkerlucioI've been giving second thoughs around the plugins, because batching also suffers from the limitations of the "wrapping pattern"#2021-06-2417:02wilkerlucioits also interesting to think about what is a good way to provide hooks for metrics like this#2021-06-2417:02wilkerluciobecause measuring single vs batch, its not the same thing, and maybe is better to have distint ways to track it#2021-06-2419:42henrikThat’s a good point, and it’s not the end of the world if it's two separate hooks. To me, it doesn't matter for the purposes of Honeycomb, since it's interesting enough to see where the app is spending a lot of time. From that point of view, it doesn't matter whether it’s batched or not. But I get that there may be other circumstances where it does matter.#2021-06-2419:48henrikWith Honeycomb, I could inject the variable of whether it's batched or not, if this distinction could be known inside the plugin definition. This would enable me to split the analysis on those that are batched and not in the Honeycomb UI. Exposing an indicator of whether the plug-in is currently running inside a batch resolver or not might be more convenient than using two distinct integration points. That would leave it up to the creator of the plug-in to decide whether this information matters to them or not.#2021-06-2422:08wilkerluciocurrently you can check if you look at the resolver, the "node" has ::pco/op-name, which you can use to lookup at ::pci/index-resolvers, them pull the config map with pco/operation-config#2021-06-2422:08wilkerluciothe major missing point is a hook to know about the batch run time#2021-06-2422:08wilkerluciothat's currenty not available, but I can add that, just like to think a bit more about the whole plugin thing, this is something I can get to this week#2021-07-0111:53wilkerlucio@U06B8J0AJ yesterday I made some changes to what wrap-resolve wraps, before it was wrapping some higher process, but now it really wraps the resolve call (as I believe you expected, which makes sense), note now you get env input instead of env node, please let me know how that fits in your game#2021-07-0111:53wilkerlucioimportant to note though, now in the case of batch, you will only get 1 call to wrap-resolve with the batch (`input` will be a collection)#2021-07-0114:59henrikThis works very well, thank you. I can now add the size of the input to the logging with Honeycomb. (i.e., "it took 59ms, but on the other hand there were 2291 inputs")#2021-06-2411:59JanosHey we have a prod service using Pathom2 and started to see some weird behavior for some queries. After reducing the query’s scope to two attributes we can see errors like this:
"Assert failed: Tried to remove node that still contains references pointing to it. Move\n      the run-next references from the pointer nodes before removing it.\n(if after-nodes (every? (fn* [p1__41590#] (not= node-id (-> (get-node graph p1__41590#) :com.wsscode.pathom.connect.planner/run-next))) after-nodes) true)"
Do you have any pointer what should we look for during investigation?
#2021-06-2412:00JanosInterestingly changing the order of the two problematic attributes in the query fixes the problem. Still we would like to understand the issue and potentially fix it#2021-06-2417:00wilkerluciohello Janos, from the error message I assume you are using reader3?#2021-06-2418:08imreThat's correct. Back when this service was implemented, there were performance tests done and based on the results the team went with async-parser + reader3#2021-06-2419:05wilkerluciothat reader is experimental in pathom 2, the planner there is stopped, if porting to reader2 makes performance too bad, them maybe better to try to use Pathom 3 (which uses a much more evolved version of the algorithm used in reader3 on Pathom 2)#2021-06-2420:38JanosIs it pathom 3 already in a shape you would recommend using it in prod?#2021-06-2420:40JanosI would be happy to migrate to that just didn't know that it's already has all the features pathom2 had#2021-06-2421:51wilkerlucioI mean, reader3 is as risk as it, there may be some internal breakages in pathom3, but the external api is mostly stable, the prod stable still is the reader2 on Pathom 2, but between reader3 and pathom 3, pathom 3 has a better algorithm, because its an evolution of reader3 (which wont be getting any updates)#2021-06-2504:56JanosI see, thanks for the info, we'll check using reader2 would solve the problem as a quick fix, if not then we could migrate to pathom3 (which we planned to do anyway)#2021-06-2516:43dehliHi, I’ve got some strange behavior that I’m trying to trace down in pathom2. I’ve got a resolver that calls out to the parser from within it. To narrow down the issue I’ve started removing keys that are in the subquery to figure out what’s causing the misbehavior. I was able to reduce the query all the way to the following and it still returns unexpectedly. To me it seems like this should be impossible. Is there any ideas for how I can further debug?
;; Subquery within the resolver
(parser env [{[:curriculum/uuid "my-uuid"] [:curriculum/uuid]}])

;; Response from above subquery
{[:curriculum/uuid "my-uuid"] {:curriculum/uuid ::p.core/not-found}}
#2021-06-2517:14wilkerluciolooks strange indeed, any chance a plugin may be doing something?#2021-06-2517:45dehliUnfortunately not. I’ve removed all plugins (other than pc/connect-plugin)and I’m still seeing that behavior.#2021-06-2517:47dehliI’m going to try to remove references to placeholders in my test and then remove the placeholder reader to see if that’s related#2021-06-2713:54wilkerlucioI tried to replicate the issue in a small setup, but the results looks as expected:
(pc/defresolver inner-call [{:keys [parser] :as env} _]
  {::pc/output [:inner-result]}
  {:inner-result (parser env [{[:foo 1] [:foo]}])})

;; define a list will our resolvers
(def my-resolvers [inner-call])

;; setup for a given connect system
(def parser
  (p/parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/reader2
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate
     ::p/plugins [(pc/connect-plugin {::pc/register my-resolvers})
                  p/error-handler-plugin
                  p/request-cache-plugin
                  p/trace-plugin]}))

(comment
  (parser {} [:inner-result]))
#2021-06-2812:00dehlithanks! it normally works as expected. it’s just when i have many resolvers nested that I get this strange behavior#2021-06-2812:00dehliif I can’t trace it down, i’ll try to come up with a reproducible example#2021-06-2815:03dehliLooks like I was able to reproduce the bug in a smaller project. I think it has to do with parallel-parser or parallel-reader b/c when I switch to using the regular reader / parser the bug goes away. Here’s the example project I setup. https://github.com/dehli/pathom2-bug/blob/main/src/dev/dehli/pathom_bug.cljs#2021-06-2816:15wilkerlucioyeah, looks off, started an issue for it https://github.com/wilkerlucio/pathom/issues/198#2021-06-2816:19wilkerluciook, I did some investigation, and I think I got it#2021-06-2816:21wilkerluciothe problem is that in the parallel case the entity gets modified upon each request#2021-06-2816:25dehliAwesome! That’s great news šŸ™‚ Thanks!#2021-06-2816:27wilkerluciojust replied on the issue with details, please let me know if the solution proposed there works for you#2021-06-2816:40dehliThanks! I think that would work. I was initially using placeholders, and the bug only started to surface as I moved to idents. Is this intended behavior for the parallel-parser?#2021-06-2816:42wilkerluciousing parser inside parser has some of this repercusions, its kinda expected#2021-06-2816:42wilkerlucioif you are using it too much, would be nice to see why, ,and if it could be avoied#2021-06-2816:43wilkerluciobut there cases (specially on pathom 2) that its just the only way (a hack way for optional and/or nested inputs)#2021-06-2816:50dehlicool! we don’t use them too often (almost always it’s due to nested inputs). pathom3 will fix that, so we are excited to make the switch šŸ™‚#2021-06-2918:24dehliHello again! I’m still seeing similar behavior (even when using the (with-entity) function you had suggested. I’ve also tried clearing out ::p/request-cache and I’m still getting a subset of my query returned in my nested parser call. Is there some other key in env that I could look at to debug? If not, I can try to find another reproducible example.#2021-06-3017:19dehliI’ve got another example showing the bug I’m seeing. For this one, I am using the with-entity function. I’ll keep playing around with it to try to better figure out what’s going on.
(ns dev.dehli.pathom-bug
  (:require [clojure.core.async :refer [go <! <!!]]
            [com.wsscode.pathom.core :as p]
            [com.wsscode.pathom.connect :as pc]
            [com.wsscode.pathom.parser :as pp]))

(defn with-entity [env ent]
  (assoc env ::p/entity (atom ent)))

(pc/defresolver one
  [{id :id}]
  {:one (str "one " id)})

(pc/defresolver state
  [{:keys [parser] :as env} {id :id}]
  {::pc/output [:state]}
  (go
    {:state (<! (parser (with-entity env {:id id}) [:one]))}))

(def parser
  (p/parallel-parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/parallel-reader
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}
                  ::pp/external-wait-ignore-timeout 100}
     ::p/plugins [(pc/connect-plugin {::pc/register [state
                                                     one]})
                  p/error-handler-plugin
                  p/trace-plugin]}))

(comment
  (<!! (parser {} [{[:id "foo"] [:one :state]}]))

  ;; Returns
  ;; {[:id "foo"] {:one "one foo" :state {}}}
  )
#2021-06-3019:48dehliIt seems to be returning empty state b/c it’s reaching the ::pp/external-wait-ignore-timeout#2021-06-3020:42dehliI think I was able to fix the bug by adding ::pp/processing-recheck-timer#2021-06-3020:42dehlialthough it does seem like a bandaid rather than a full solution#2021-06-3020:43wilkerlucioyeah, I'm taking a deeper look, it doesn't look very good#2021-06-3020:43wilkerluciothe issue is that the parallel processor has all these "tricks" add for performance tuning over time#2021-06-3020:44wilkerlucioand these examples are exposing their sensitivity for recursive calls#2021-06-3020:47dehliI see. That lines up with what I’m seeing on my end as well. Our recently added resolvers utilize recursion which must be why we’re starting to see some unexpected behavior.#2021-06-3020:51wilkerluciodid you recently changed to parallel parser? or recently add recursive calls like this?#2021-06-3020:53dehliwe’ve been using parallel parser but recently added recursive calls like this#2021-06-3020:56wilkerlucioand the reason you use the parallel parser is related to long attribute dependencies? or because of processing many entities in parallel? ou maybe both?#2021-06-3020:58dehliwe’re using the parallel parser for the second reason (needing to execute many entities in parallel)#2021-06-3020:58dehlii guess the first as well#2021-06-3020:59dehliwe saw pretty good performance improvements when we switched to parallel#2021-06-3020:59wilkerluciogotcha, we can try to hack a bit on the parallel and "reset" the things that need reset on the recursive call, we may be able to make this work for now#2021-06-3021:01dehliya, i was looking through the env map to try and figure out what could be reset to give it a ā€œfreshā€ view#2021-06-3021:03dehliadditionally the bandaid might be enough as well until we make the switch to pathom3. it seems that the parallel parser can be implemented much simpler with the new architecture#2021-06-3021:06wilkerlucioyes, parallel processing in pathom 3 should be much simpler#2021-06-3021:06wilkerlucioI'm thinking of doing something in pathom 2 that I did for pathom 3 regarding env#2021-06-3021:06wilkerluciothere I just save a "backup" version of the env on entry, so you can restore to it#2021-06-3021:06wilkerluciobut may not be enough, because of some mutable state things, like active-paths#2021-06-3021:11wilkerluciook, I find something that works with your example:#2021-06-3021:11wilkerlucio
(defn with-entity [env ent]
  (assoc env ::p/entity (atom ent)))

(pc/defresolver one
  [{id :id}]
  {:one (str "one " id)})

(defn reset-env [env]
  (assoc env
    :com.wsscode.pathom.parser/active-paths (atom #{})
    :com.wsscode.pathom.parser/waiting #{}
    :com.wsscode.pathom.parser/key-watchers (atom {})
    :com.wsscode.pathom.core/entity-path-cache (atom {})))

(pc/defresolver state
  [{:keys [parser] :as env} {id :id}]
  {::pc/output [:state]}
  (go
    {:state (<! (parser (-> env
                            (reset-env)
                            (with-entity {:id id})) [:one]))}))

(def parser
  (p/parallel-parser
    {::p/env     {::p/reader                        [p/map-reader
                                                     pc/parallel-reader
                                                     pc/open-ident-reader
                                                     p/env-placeholder-reader]
                  ::p/placeholder-prefixes          #{">"}
                  ::pp/external-wait-ignore-timeout 100}
     ::p/plugins [(pc/connect-plugin {::pc/register [state
                                                     one]})
                  p/error-handler-plugin
                  p/trace-plugin]}))
#2021-06-3021:12wilkerlucioI can make reset-env part of the Pathom API, so we can move it with the library (and if we find more things to reset, an update will fix)#2021-06-3021:13dehliawesome! ya, i’ll give that a go. I have a quick question about the processing-recheck-timer how does it know it’s ā€œstuckā€? does that just mean the resolver is still executing? Trying to determine what’s a good value so that I don’t retry database calls, etc.#2021-06-3021:18wilkerlucioIm reading again on the code to remember what thats about#2021-06-3021:21wilkerluciofor reasons I don't remember now, the parser may get stuck in some part of the process, this is probably due to the channel madness going on in the orchestration there, looking at the code, its a kinda of "timeout" case, the parallel parser will walk all children of the AST, and try to make sure every attribute is "started" (not every single attribute will trigger a process, for each process pathom knows all "expected" attributes, and will skip those during the scan in case a previous attribute ended up hitting it in the process)#2021-06-3021:22wilkerlucioso after start everything, it waits for some response, to merge as they come#2021-06-3021:22wilkerluciobut if it is stuck, that will not finish, this is where the process-recheck-timer enters, it its like a timeout case for this step, and if the code reaches it, it will re-trigger the process for the attributes that still pending#2021-06-3021:23wilkerluciomakes sense?#2021-06-3021:24dehliya, that makes sense! thanks so much! i’ve also gotten your reset-env function added to our subquery function so i think that will fix the issues we’ve run into. will test it out and let you know!#2021-06-3021:25wilkerlucioa little polishing on the api:#2021-06-3021:25wilkerlucio
(defn with-entity [env ent]
  (assoc env ::p/entity (atom ent)))

(pc/defresolver one
  [{id :id}]
  {:one (str "one " id)})

(defn reset-env [env]
  (assoc env
    :com.wsscode.pathom.parser/active-paths (atom #{})
    :com.wsscode.pathom.parser/waiting #{}
    :com.wsscode.pathom.parser/key-watchers (atom {})
    :com.wsscode.pathom.core/entity-path-cache (atom {})))

(defn call-parser [{:keys [parser] :as env} entity tx]
  (parser
    (-> (reset-env env) (with-entity entity))
    tx))

(pc/defresolver state
  [env {id :id}]
  {::pc/output [:state]}
  (go
    {:state (<! (call-parser env {:id id} [:one]))}))
#2021-07-0114:41dehliThanks again for all your help! Have deployed with the changes and the unexpected behavior is now gone šŸ™‚#2021-07-0114:55wilkerluciogreat to hear! šŸ™‚ later I'll put the reset-env at the library, so you can use from it#2021-06-3015:35tony.kayHey @wilkerlucio, how's Pathom 3 coming along? I'm working on RAD recently, and the db adapters depend pretty heavily on Pathom. Wondering if there's going to be a compat layer.#2021-06-3021:32wilkerlucioalpha is very close šŸ™‚, with some release plans, hold tight!#2021-07-0318:26zZMathSPHas anyone ever tried using Pathom on Leiningen?#2021-07-0320:26wilkerlucioPathom 2 should work fine, Pathom 3 is a bit trickier because it currently only supports git as a deployment option#2021-07-0322:42zZMathSPThanks!#2021-07-0501:29nivekuilseems I can't upgrade past https://github.com/wilkerlucio/pathom3/commit/4ddc0b765ceeecb0587cc7e031cb834bdc3c0444 or else queries that hit a few of my resolvers error with
com.wsscode.pathom3.connect.runner.async/run-node!                 async.cljc:  328
java.lang.IllegalArgumentException: No matching clause: :com.wsscode.pathom3.connect.planner/node-unknown
any breaking changes expected? I was on the commit right before that 7e61113f1818e63d9528e61344a79bca20db969c and that's also the last one that works.. weird
#2021-07-0503:59wilkerluciothere was a breaking change regarding the ::pcr/wrap-resolve extension, do you have a plugin using that?#2021-07-0504:02wilkerluciobut may be a bug, seems an old change, can you make a repro case?#2021-07-0505:11nivekuilhmm.. there's an and node that references 11 but there is no node-id 11#2021-07-0505:12nivekuilso pcp/get-node returns nil for that node-id#2021-07-0505:19nivekuiloh, 7e6 is actually a different branch from 4ddc entirely.. github makes it look linear šŸ˜•#2021-07-0505:37nivekuilterrible hack I know, but I just set pcp/node-kind to default to ::node-resolver instead of ::node-unknown and it works on master#2021-07-0505:47nivekuilI can't figure out a repro, happens even without any plugins#2021-07-0513:05wilkerluciocan you try setting ::pcp/optimize-graph? false on your env and see if you still see the issue?#2021-07-0513:10wilkerlucioand to confirm, you see that on master?#2021-07-0520:30nivekuiloptimize-graph? seems to have no effect. still just seems to loop through a few run-and-node! calls before blowing up. yes, all this is done on master#2021-07-0605:48wilkerlucioa repro would be great, so can try to figure what's causing the issue#2021-07-0513:26mitchelkuijpersWhat is the best way to handle exposing errors with the remove-stats-plugin cominbed with the attribute-errors-plugin? I tried following this: https://pathom3.wsscode.com/docs/built-in-plugins#remove-stats but when I add the remove-stats-plugin before the attribute-errors-plugin I lose all errors and will just return an empty map for a failing resolver. I alsno noted a deprecation message. I already turned off that transit exposes metadata so maybe I shouldn't care#2021-07-0514:17mitchelkuijpersAh I fixed it by adding: :com.wsscode.pathom3.connect.runner/run-stats-omit? true to the env#2021-07-0514:17mitchelkuijpersAh then I still have the problem that when a resolver fails the parser returns an empty map#2021-07-0514:41wilkerluciojust taking a look at it now#2021-07-0514:42wilkerlucionot ideal, but one option is to just not encode meta on the transit as a workaround for now#2021-07-0514:44wilkerlucioand I deprecated the remove-stats, and renamed that omit flag#2021-07-0514:45wilkerlucio
- `::pcr/run-stats-omit?` => `::pcr/omit-run-stats?`
- `::pcr/run-stats-omit-resolver-io?` => `::pcr/omit-run-stats-resolver-io?
`
#2021-07-0514:45wilkerluciosorry the breakages, during when it still on time, planning to release an alpha this week#2021-07-0521:26mitchelkuijpersNo worries at all it is expected, pathom3 is working very solidly already#2021-07-0514:53wilkerluciohello everyone, for anyone using Pathom 3, there is a new option ::pcr/fail-fast? true that you can use, this way exceptions get thrown up from resolvers/mutations immediatly#2021-07-0520:34souenzzoCan we fail fast just mutations?#2021-07-0602:42wilkerlucionot at this point, lets talk about it: https://github.com/wilkerlucio/pathom3/discussions/65#2021-07-0605:02wilkerlucio#2021-07-0612:50Bjƶrn EbbinghausI wrote a plugin, that checks before each resolver whether each entry in the input of said resolver is allowed. If yes, continue to resolve. If no, do necessary checks and if access is allowed, cache the input entry. Now I would like to hear feedback from people more experienced than me, if I have done something stupid or if I could do something better. https://gist.github.com/MrEbbinghaus/eb14937a0b0909530cbfa85c26c79139 Thanks. šŸ™‚#2021-07-0616:43MatthewLispHello Pathom users, I'm a bit new to EQL, and I'm wondering if it's possible to provide some specific data through eql for pathom This example using placeholders is from Pathom3 website
(pco/defresolver full-name [{::keys [first-name last-name]}]
 {::full-name (str first-name " " last-name)})
 
(def env (pci/register full-name))
 
(p.eql/process env
 [{'(:>/bret {::first-name "Bret" ::last-name "Victor"})
 [::full-name]}])
; => {:>/bret {:com.wsscode.pathom3.docs.placeholder/full-name "Bret Victor"}}
What i want to know is, can i say via EQL that the ::first-name or ::last-name, is actually an attribute that pathom should look for:
(p.eql/process env
 [{'(:>/bret {::first-name ::first-name/bret ::last-name ::last-name/bret})
 [::full-name]}])
In this case, i want to say that ::first-name/bret and ::last-name/bret are attributes that some other resolvers produces
#2021-07-0618:15Bjƶrn EbbinghausI am honestly not really sure what you are asking... You can provide constant data, if that's what you are after?
(pco/defresolver produce-bret []
  {::first-name "Bret"
   ::last-name "Victor"})

(pco/defresolver full-name [{::keys [first-name last-name]}]
 {::full-name (str first-name " " last-name)})

(p.eql/process env
  [::full-name])

=> {::full-name "Bret Victor"}

(p.eql/process env
  [{:>/bret [::full-name]}])

=> {:>/bret {::full-name "Bret Victor"}}
#2021-07-0821:39lgesslerhi, I'm having an embarrassingly simple issue with pathom 2: it's my first time implementing a to-many join in my resolvers, and I'm getting an error Don't know how to create ISeq from: clojure.lang.Keyword when pathom attempts to follow the to-many join. I have a declared pc output :foo/to-many-join, and my output looks like {:foo/id ... :foo/to-many-join [[:bar/id 1] [:bar/id 2]]. Is there something wrong with the shape of my data for :foo/to-many-join?#2021-07-0821:41lgesslerI do have ::pc/open-ident-reader in my parser's ::p/reader , which I think is necessary for the ident-style joins#2021-07-0822:36wilkerluciohello, yes, the output shape is wrong, a to-many must have a collection of maps, not idents#2021-07-0822:37wilkerluciothis is in contrast with Fulcro, in Fulcro normalized DB it uses the idents to ref things, but that's not how Pathom works#2021-07-0822:37wilkerlucioin your case you need to return something like: {:foo/id ... :foo/to-many-join [{:bar/id 1} {:bar/id 2}]#2021-07-0822:37wilkerlucioso you have the initial data for pathom to extend on top#2021-07-0822:39lgesslergot it, thank you!#2021-07-0822:40lgesslerfollow-up question... before I checked back here for your messages I just finished refactoring my code to use p/parallel-parser with pc/parallel-reader and pc/mutate-async and the fulcro-style ident seems to work?#2021-07-0822:40lgesslerI'm guessing this is unofficial behavior which may break in the future, right?#2021-07-0822:52lgessleranyway... i'll just use hashmaps now#2021-07-0823:13wilkerluciobe careful with parallel parser though, there only a few cases in which its a good option for general users#2021-07-0823:14wilkerluciomost of the time is slower and way more complex, not worth until you can really leverage a lot parallelism (measure your cases if you wanna check)#2021-07-0823:14wilkerluciobut in general, better to stick with the serial one#2021-07-0903:40lgesslerthanks, didn't realize that. back to reader2 for now then#2021-07-0910:18cyppanI’m playing with Pathom 3 EQL queries and I don’t understand why you get • '(:_>_/bret {::first-name "Bret" ::last-name "Victor"}) as a ::pco/input in your resolver • '(::todos {::todo-done? false}) in (pco/params env) instead I would expect the same behavior for both the EQL parameter and the placeholder parameter, do I miss something?#2021-07-0911:19souenzzo@U0CL38MU1 it's about placeholders https://pathom3.wsscode.com/docs/placeholders/ when you use :>/anything, its parameter will be merged into the "input" > are special keywords. In any other context, it will end in p/params#2021-07-0914:00cyppanyes I understand the ā€œwhatā€ I was just wondering about the ā€œwhyā€#2021-07-0914:52souenzzoPlaceholders are the way to the client provide inputs to the server. Params are to customize a specific attribute resolver#2021-07-0915:15cyppanbut params can be provided by the client too#2021-07-0915:17cyppan
(def client-provided-toto-filters {::todo-done? false})
`(::todo ~client-provided-toto-filters)
But it might not be idiomatic to do that way...?
#2021-07-0915:29souenzzoparams will not "connect" input will "connect"#2021-07-0915:31cyppanaaah ok yes I understand ! thanks šŸ™‚#2021-07-0916:55souenzzotbh they overlap a lot. I usually always prefer inputs Just swap to params if fall into some problem#2021-07-1000:51fabraohello all, what is the difference using : or :: ?#2021-07-1000:52fabraofor key difinition?#2021-07-1000:52lsenjov:foo resolves to :foo, ::foo resolves to :*ns*/foo#2021-07-1000:52lsenjovSo if you're in the bar.baz namespace, then ::foo resolves to :bar.baz/foo#2021-07-1000:53lsenjovIt also allows aliasing, so if you have the alias [bar.baz :as b] then ::b/foo resolves to :bar.baz/foo#2021-07-1000:54fabraoso how do I query this?
(pc/defresolver cursos-alura [env _]
                {::pc/output [{::cursos-alura [:curso/id]}]}
                {::cursos-alura [{:curso/id 1}
                                 {:curso/id 2}]})
#2021-07-1000:54lsenjovWhat namespace is it in?#2021-07-1000:54fabraois it matter?#2021-07-1000:54fabrao
(parser {} [{::cursos-alura [:curso/id]}]) does not work
#2021-07-1000:54lsenjov...yes, that's the point I'm getting at#2021-07-1000:54lsenjovWhat namespace is the defresolver in?#2021-07-1000:55fabraoaplicacao.resolvers.alura.cursos#2021-07-1000:55lsenjovThen it's :aplicacao.resolvers.alura.cursos/cursos-alura#2021-07-1000:55lsenjovOr ::cursos/cursos-alura if you set up a cursos alias#2021-07-1000:56fabraounderstando, let me try here#2021-07-1000:56fabrao
(parser {} [{::cursos/cursos-alura [:curso/id]}])
#2021-07-1000:56fabrao
{:aplicacao.resolvers.alura.cursos/cursos-alura [{:curso/id 1} {:curso/id 2}]}
#2021-07-1000:57fabraono way to resolve with alias?#2021-07-1000:58lsenjovHave you set up an alias for cursos in your ns declaration?#2021-07-1000:58fabraoin registry I used [aplicacao.resolvers.alura.cursos :as cursos]#2021-07-1000:59fabraoso, yes I used it#2021-07-1000:59lsenjovWait, that looks like it's resolving fine?#2021-07-1001:00fabraoI thougth that response shoud be {:cursos/cursos-alura [{:curso/id 1} {:curso/id 2}]}#2021-07-1001:00lsenjovNo, ::cursos/cursos-alura literally equals :aplicacao.resolvers.alura.cursos/cursos-alura#2021-07-1001:01fabraoyes sure, but is there any way to use the alias?#2021-07-1001:01lsenjovThe :: makes it resolve the namespace#2021-07-1001:01lsenjov::cursos/ resolves the namespace, and does not equal :cursos/#2021-07-1001:02fabraoif I use it with the same namespace as the registry?#2021-07-1001:03lsenjovI'm not sure I follow what you're trying to achieve#2021-07-1001:03lsenjovIf you want it to be :cursos/cursos-alura, then you should make that the key of your defresolver#2021-07-1001:04fabraoI want to be like this#2021-07-1001:06fabrao"If you want it to beĀ `:cursos/cursos-alura`, then you should make that the key of yourĀ `defresolver` " -> Understood#2021-07-1001:09fabrao@lsenjov Thank you#2021-07-1001:49fabraohow do I pass parameter in eql query?#2021-07-1001:49fabraoI saw this [(::instruments {:sort :instrument/brand})] but it does not work#2021-07-1002:43wilkerluciowhen running from Clojure you need to quote it, otherwise it tries to invoke the keyword: ['(::instruments {:sort :instrument/brand})]#2021-07-1002:49fabraoohhhhhhhhhhhhhhhhhhhhhuuuu#2021-07-1002:50fabraoIĀ“m a dumb guy#2021-07-1002:50fabraothank you @U066U8JQJ#2021-07-1001:50fabraogiving me Invalid excpression#2021-07-1004:11wilkerlucio#2021-07-1119:35Bjƶrn EbbinghausHow much work is it to migrate from v2 to v3?#2021-07-1121:50wilkerluciodepends on what you use from Pathom 2, resolvers are mostly compatible (in terms of definition, the major difference is that inputs are now EQL instead of sets). plugins also require change. I'm going to start working on a "portability library" this week, I wanna see how automatically we can make it, the idea is to have a different namespace that you require and you use the same Pathom 2 interface to use Pathom 3 implementation#2021-07-1116:52wilkerluciohello everyone, I'm thinking about some big changes for Pathom 3 error handling, changing it from "tolerant first" to "strict first", that means errors will get throw much sooner, I wrote more details of what I'm thinking about at https://github.com/wilkerlucio/pathom3/discussions/65#discussioncomment-990684 and would love your feedback#2021-07-1121:51wilkerlucio#2021-07-1208:10cyppanAwesome thanks šŸ™‚ After downloading it I do get an error though (I’m on the last commit of pathom and on the last version of the pathom-viz connector)
Mon Jul 12 10:01:42 CEST 2021 [worker-2] ERROR - POST /
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: No implementation of method: :-operation-config of protocol: #'com.wsscode.pathom3.connect.operation.protocols/IOperation found for class: com.wsscode.pathom3.connect.operation.Resolver
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
	at clojure.core$deref_future.invokeStatic(core.clj:2304)
	at clojure.core$deref.invokeStatic(core.clj:2324)
	at clojure.core$deref.invoke(core.clj:2310)
	at com.wsscode.pathom.viz.ws_connector.impl.http_clj$handler.invokeStatic(http_clj.clj:46)
	at com.wsscode.pathom.viz.ws_connector.impl.http_clj$handler.invoke(http_clj.clj:37)
	at com.wsscode.pathom.viz.ws_connector.impl.http_clj$start_http_server_BANG_$fn__85399.invoke(http_clj.clj:74)
I can make an issue maybe?
#2021-07-1213:18wilkerluciothat's a strange error, because the type is right, so Im not sure why it can't dinf the operation-config protocol, can you make a repro case?#2021-07-1617:41cyppanI’ve changed the versions of pathom3 (2021.07.10-1-alpha) and the connector (2021.07.15-1) to the latest Clojars releases and I don’t reproduce everything work fine, it must have been a problem on my end..!#2021-07-1216:51genekimI was inspired by @wilkerlucio response to someone asking about how to write Pathom queries, and attempted to write one myself. https://clojurians-log.clojureverse.org/pathom/2019-10-22 I found myself immediately wanting to implement it by composing: 1) get all the stories (a query I had already written [:story/all-stories], and then 2) filter them according to the given criteria. I’m delighted I got something working, but for expediency, did something quite ugly — I called parser from inside the resolver, using resolve to avoid the circular dependency, fetching parser/parser and parser/config ,which I had to deref to avoid Don't know how to create ISeq from: clojure.lang.Var error. It works, but I’m guessing there’s a better way to retrieve an EQL query? Or compose resolvers? Thx!#2021-07-1216:52genekim#2021-07-1216:53wilkerluciohello @genekim, for this would be better if you use :story/all-stories as the input in the story-search-resolver, this way you don't have to call the parser from inside#2021-07-1216:54wilkerlucioin Pathom 3 this story is better, because you can do nested inputs on Pathom 3, this means your resolver could have: ::pco/input [{:story/all-stories [:story/id :story/author :story/title]}]#2021-07-1216:56wilkerluciohttps://pathom3.wsscode.com/docs/resolvers/#nested-inputs#2021-07-1216:55wilkerluciothis way, even if the query is complex (for the nested part under all-stories) Pathom would resolve it first and give the exact input requested#2021-07-1216:55wilkerluciobut if you need the nested details in Pathom 2, them doing a recursive call to the parser is the only way#2021-07-1216:59genekimHoly cow, @wilkerlucio — ::pco/input. That’s amazing. Reading docs on this right now. To confirm my understanding: this would make :story/all-stories available from inside the search resolver, and then I could filter on this?#2021-07-1216:59genekim(pathom2, as I’m using Fulcro)#2021-07-1217:15genekimTrying to wrap my head around 1) how to get all the stories in the env. This is the current query that’s working, where search text comes in as a parameter.
['(:search-results/stories {:search/search-query "search text abc!"})])
to…. argh, can’t quite figure out where the :story/all-stories would go? I’m sure it’s not this…
[['(:search-results/stories {:search/search-query "search text abc!"})
  :story/all-stories])
Thx for your help here, @wilkerlucio!
#2021-07-1218:24wilkerluciohere is a complete example:
(pc/defresolver all-items []
  {::pc/output [{:all-items [:id :name]}]}
  {:all-items
   [{:id 1 :name "Aa"}
    {:id 2 :name "Ab"}
    {:id 3 :name "Bb"}
    {:id 4 :name "Cc"}]})

(pc/defresolver search [env {:keys [all-items]}]
  {::pc/input  #{:all-items}
   ::pc/output [{:search-results
                 [:id :name]}]}
  (let [search (-> env :ast :params :search)]
    {:search-results
     (filterv #(str/starts-with? (:name %) search) all-items)}))

(comment
  (parser {} ['(:search-results {:search "A"})])
  (parser {} ['(:search-results {:search "B"})]))
#2021-07-1218:24wilkerlucio(made in Pathom 2)#2021-07-1218:24wilkerluciothe second argument of the resolver is the input#2021-07-1221:00genekim@wilkerlucio Wow. That is so freaking cool. Absolutely freaking marvelous. THANK YOU!!#2021-07-1221:17genekimThe more I study this, the more I feel like I understand Pathom, and the more I’m in awe of it. Freaking awesome, @wilkerlucio!#2021-07-1221:40wilkerlucioone way to look at is like pattern matching, but based on shapes of data (expressed in EQL)#2021-07-1222:03genekimIt’s awesome — last question on this, before I call this done! Inspired by your response to that person in 2019, I want to put the search-text and the # of matches into the return result. Is this pc-output shape specified correctly?
(pc/defresolver story-search-resolver [env _]
       {; no ::pc/input
        ::pc/output [:search-results/search-text
                     :search-results/matched
                     {:search-results/stories
                      [:story/id :story/title :story/author :story/content]}]}
       (do
         (log/warn "*** story-search-resolver")
         (handle-search env)))))
The return result from inside the resolver is shown in the screenshot, which looks good! It’s of the shape that I want/expect: But the actual return result from the resolver for this query is… confusing.
['(:search-results/stories {:search/search-query "re-frame"})
     [:search-results/matched :search-results/search-text
      {:search-results/stories [:story/author]}]])

; =>

{:search-results/stories (#:story{:id "K3Y7GLlRfaBDsUWYD0WuXjH/byGbQnwaMWp+PEBoUZw=_16f347d1381:18e306:512e2118",
                                  :author "Arne Brasseur",
                                  :title "Advent 2019 part 23, Full size SVG with Reagent",
                                  :content "XXXXXX"
),
 [:search-results/matched :search-results/search-text #:search-results{:stories [:story/author]}] #:search-results{:matched :search-results/search-text}}
1. I only wanted the story/author to be retrieved, but all the :story fields were retrieved. 2. The search-results/matched | search-text don’t show up as keys in the return map, but are nested inside a vector (or worse. šŸ™‚ The shape I really want is:
#:search-results{:matched 0 :search-text "re-frame" :story #:story{:id :author :content :title}}
Thx again!
#2021-07-1314:27wilkerluciohello#2021-07-1314:27wilkerlucioI just took a look, the issue is in you query I believe#2021-07-1314:27wilkerlucioyou see. there is no connection in the query between :search-result/stories and the :search-result/matched ...#2021-07-1314:27wilkerlucioits missing a map connecting then#2021-07-1314:28wilkerlucioshould be like this:
[{'(:search-results/stories {:search/search-query "re-frame"})
  [:search-results/matched :search-results/search-text
   {:search-results/stories [:story/author]}]}]
#2021-07-1318:09genekim@wilkerlucio Holy cow. THANK YOU! (And a huge facepalm, seeing the missed join.). Can’t wait to see this run!#2021-07-1616:16markaddlemanShould we expect the planner to discover cycles in the attribute graph? I accidentally this situation and ended up with a StackOverflowError. I'm not too surprised that the planner doesn't detect it but I thought I would ask.#2021-07-1616:26wilkerluciothe planner does detect cycles https://github.com/wilkerlucio/pathom3/blob/master/test/com/wsscode/pathom3/connect/planner_test.cljc#L440#2021-07-1616:27wilkerlucioit could be stack overflowing more because of too deep branching#2021-07-1616:29wilkerluciocan you make a repro?#2021-07-1616:29wilkerluciothere is another debug method using the snapshots#2021-07-1616:30wilkerlucioif you use pcp/compute-plan-snapshots, even if there is an exception it still captures the snapshots as it goes, so you can see the steps before the failure#2021-07-1703:57wilkerlucioI just remembered that I had a case like that in Pathom 2, and a simple thing that may work for your case is increasing the stack size on the JVM#2021-07-1703:57wilkerlucio(in case its not a cycle loop of some kind)#2021-07-1704:33markaddlemanThanks. I'll look into that. In my particular case, the resolver has an input join where the joined attributes are also output from the same resolver.#2021-07-1704:34markaddlemanI haven't tried a repro case yet but Im pretty sure that no acyclic plan can be created#2021-07-1704:38wilkerluciodoes it has nested inputs?#2021-07-1713:00markaddlemanHere's the resolver input/output declaration:
{::pco/input    [:event :portfolioKey :appKey :event/type {:aliased.event/events [:event/name-expr :event/filter-expr]}]
   ::pco/output   [:event/filter-expr]
   ::pco/priority 1}
#2021-07-1713:02markaddlemanIf I remove :event/filter-expr from the nested input, there is no stack overflow.#2021-07-1713:02markaddlemanI'm working on a repro case now#2021-07-1819:12wilkerluciothanks to your example I was abre to reproduce it here, and yes, Pathom is missing cycle detention when its about nested inputs#2021-07-1821:07wilkerluciohttps://github.com/wilkerlucio/pathom3/issues/72#2021-07-1821:17wilkerlucioFixed on main#2021-07-1821:47markaddlemanOh great! You just saved me an hour of creating a repro case. Thanks 😊#2021-07-1822:05wilkerlucioplease note main has a big breaking change related to the strict requests, other than that it should be good to go#2021-07-1822:06wilkerluciocheck changelogs for more details on the things at it#2021-07-1822:25markaddlemanThanks. I have played with strict and discovered that I have some application fixes. I think I'll just turn off strict mode for now#2021-07-1618:30mauricio.szaboHi, @wilkerlucio! Recently, I was investigating what I thing it's a memory leak on the Chlorine project and found this situation below. When I tried to go deeper, every node that consumes more memory points to $com$wsscode$pathom3$connect$planner$compute_missing_chain_deps$$. I'm only asking if you have anything in mind, like, "ok, maybe I'll have to refactor this code in the future" so I can try to check if that's the code that's giving problems, etc... šŸ™‚ (Just to be clear: I'm not sure that Pathom is at fault here yet, I'm just debugging šŸ˜„)#2021-07-1700:26wilkerlucioone thing about planning, did you setup a persistent cache for planning? that can reduce a lot the cost of planning, giving queries are usually consistent (same queries, which means same plan result, even if the input/output data is different)#2021-07-1701:05mauricio.szaboNot really. In fact, wouldn't setup a persistent cache take a bigger hit on memory?#2021-07-1702:20wilkerluciocould be, but it would avoid the process of compute-run-graph completly (just enter, read cache, exit), I expect that to be a good tradeoff for most usages#2021-07-1702:45mauricio.szaboA question: if I have 2 resolvers for the same data, and one sometimes fails and sometimes don't, do the persistent cache for planning that that into account when resolving things?#2021-07-1702:46mauricio.szabo(the one that fails have higher priority)#2021-07-1703:26wilkerluciothe plan is a variable of indexes + query + available data shape (not the data specifically, just the shape of it, for instance, if you have the data {:foo "bar" :baz {:deep "data"}} the shape of that is {:foo {} :baz {:deep {}}}, so similar data like {:foo "other thing" :baz {:deep "different here"}} will have the same shape, although the values are different)#2021-07-1703:26wilkerlucioso, if something fails sometime,s that's always after the planning is done#2021-07-1703:26wilkerluciobecause the plan always have every possible option#2021-07-1703:27wilkerlucioin other words: the plan is the graph before running it#2021-07-1703:27wilkerlucioand the plan never changes after it start running#2021-07-1703:36wilkerlucioso for the case of 2 resolvers for the same data, the plan will always include both options, connected via an OR node#2021-07-1704:04wilkerlucioexample setup https://pathom3.wsscode.com/docs/cache/#planning-computation#2021-07-1621:46nivekuilhas anyone thought of using pathom to replace integrant? connecting integrant pieces together feels too manual#2021-07-1700:38nivekuilhttps://gist.github.com/nivekuil/1e5320b69f02cd55f39e6b9be8c48bf6#2021-07-1700:59souenzzo@U797MAJ8M yes https://github.com/molequedeideias/summon#2021-07-1701:00nivekuilyou replied too late šŸ˜ž already invested in this thing#2021-07-1701:01nivekuilI think my thing is a little different actually since it's an integrant-like API instead of component. do you use summon in prod anywhere?#2021-07-1701:02souenzzoTo be honest, I deprecated this library. The idea is nice, but deal with "smart connect" AND "state management" at the same time is really complicated.#2021-07-1703:54wilkerlucio@U797MAJ8M interesting way you got for the halting, one suggestion, you could put done as part of the env instead of a global variable (an atom at env), so you could control it better, and maybe add a way to avoid starting an already started system (or maybe an auto-restart in that case? by halting everything and starting over)#2021-07-1704:08nivekuilI made a few changes actually, I think it's actually very ergonomic
(nx/bind! server [{::keys [ring-handler aleph-opts executor]}]
  {::nx/halt #(.close %)}
  (start-server ring-handler (assoc aleph-opts :executor executor)))

(nx/bind! executor []
  {::nx/halt (fn [x] (.shutdown x)
               (.awaitTermination x 15 java.util.concurrent.TimeUnit/SECONDS))}
  (utilization-executor 0.9 256 {}))

(nx/init {:app.server/ring-handler  (reitit.ring/create-default-handler)
          :app.server/aleph-opts {:port 1234}}
         [:app.server/server])
to start an aleph server + executor
#2021-07-1704:09nivekuilproblem is I can only call nx/init in the same namespace as the one in which I define the resolvers (`nx/bind!`) because I don't know how to get pathom to know what start-server , inside the :resolve fn, is at resolver run time, to the macro it's just a symbol like the ones from pco/input#2021-07-1704:21wilkerluciowhat if you put start-server on the env?#2021-07-1704:22nivekuilhow do you mean? right now it comes from [aleph.http :refer [start-server]]#2021-07-1704:22wilkerluciolike
(pco/defresolver resolver [{::keys [start-server]} _]
  (start-server))

(p.eql/process
  (-> {::start-server (fn [] "implementation")}
      (pci/register resolver))
  query)
#2021-07-1704:22nivekuilideally the macro body feels exactly like defresolver, i.e. a closure#2021-07-1704:23wilkerlucioare you familiar with the resolver taking 2 arguments?#2021-07-1704:25nivekuilyeah, but the defresolver body doesn't have to be a pure function of env + params right#2021-07-1704:27wilkerluciomostly they should be, caching and logging are common exceptions#2021-07-1704:28wilkerlucioto what end you are considering the pure sense of it?#2021-07-1704:28nivekuili.e. you can close over vars in the namespace, like from :refer#2021-07-1704:29nivekuilyou don't have to put them into env to use them inside the resolver, though maybe that is a good idea? looking through my code to see how often that even happens#2021-07-1704:29wilkerluciocorrect, and most of the time won't be in the env#2021-07-1704:30wilkerlucioits nice to put on the env to create some indirection, in case you want to fill that up later#2021-07-1704:31wilkerluciomost common when writing some sort of generic resolver, or to place things like database connections, this gets a bit meta in your case because you are building the config itself on top of that#2021-07-1704:32wilkerlucioare you using the result of this setup as config for another Pathom thing, or as a stand alone system build up / shut down?#2021-07-1704:33nivekuilI actually have no idea how pathom even manages do to this.. how does the runner know to run the :resolve fn as if it were in the same namespace as it's defined?#2021-07-1704:33nivekuilI want to replace integrant completely, so stand alone#2021-07-1704:35wilkerluciothe fn binding happens when we make the resolver#2021-07-1704:35wilkerluciolooking at the resolver form:
(pco/resolver 'foo
  {::pco/output [:foo]}
  (fn [env input]
    ...))
#2021-07-1704:36wilkerluciothe resolver stores that fn defined there, so the bindings are all in place#2021-07-1704:36wilkerluciomakes sense?#2021-07-1704:44nivekuilahh, finally got it. that's a good hint, the eval needs to be done in resolver instead of defresolver#2021-07-1704:46nivekuilbasically I was calling eval at runtime instead of defresolver time.. oops#2021-07-1704:50nivekuilgoing back to what you said about putting done in env, can I dynamically modify env from a resolver call in pathom3?#2021-07-1704:50nivekuilI know there's a ::p/env in p2 but don't know what it is in p3#2021-07-1704:55nivekuiland as far as not starting an already started system, I think resolver cache might be the solution?#2021-07-1705:18wilkerlucioyou can't modify env, but you can have mutable things on it#2021-07-1821:17wilkerlucioFixed on main#2021-07-1722:56nivekuilhow does resolver cache work when you change the resolver body itself, i.e. the fn on :resolve ? does it invalidate the cache? docs say it's only keyed by name+input+params.. would it make sense to cache by :resolve instead of name?#2021-07-1814:43wilkerlucioare you modifying the resolver body after registering it?#2021-07-2002:08wilkerlucio#2021-07-2006:43Vincent CantinI would like to make a data query from the front end to the backend where; • data A is resolved, • then based on A, derive a boolean value (same as the condition in a if) which will branch between either resolve data B or data C. (B and C are of totally different types) Is it possible to do something like that inside 1 EQL query?#2021-07-2007:58Vincent CantinI found something close, I will try it tomorrow. https://pathom3.wsscode.com/docs/eql#union-queries#2021-07-2012:01wilkerluciooptional inputs is another way to handle that#2022-10-0713:06Eric Dvorsak+1 on optional input that is how I avoided a union query in fulcro https://clojurians.slack.com/archives/C68M60S4F/p1665147250627689?thread_ts=1665146697.279039&amp;cid=C68M60S4F#2021-07-2011:31fabraohello all, how do I solve this?
(let [data "18/07/2021"]
    (geral/parser
		configuration
     ['{(:cursos-alura {:dt-inicio data :dt-fim data}) [:email :termino :curso :esforco :nome]}]))
I got dt-inicio and dt-fim nil in resolver?
#2021-07-2013:01dehliThe issue is that you’re using ' instead of backquote. It should look like:
[`{(:cursos-alura {:dt-inicio ~data :dt-fim ~data}) [:email :termino :curso :esforco :nome]}]
#2021-07-2013:02dehlinotice that you use the ~ to escape the quoting#2021-07-2013:02dehliHere’s a SO question with a better explanation šŸ™‚ https://stackoverflow.com/questions/17800917/clojure-difference-between-apostrophe-and-backtick#2021-07-2013:23fabrao@U2U78HT5G You are the best, thank you, you saved my day#2021-07-2122:48caleb.macdonaldblackHow do I add a paramater such as a database connection into the env for pathom3?#2021-07-2122:53caleb.macdonaldblackOh it turns out I just assoc it in the env#2021-07-2203:16nivekuildocs seem incomplete for https://pathom3.wsscode.com/docs/plugins/#pcrwrap-entity-ready#2021-07-2300:21wilkerlucioFixed, the call had too many things, that ext point just gets env#2021-07-2203:19nivekuilnow that attribute errors are default, how can they be turned off?#2021-07-2203:25nivekuilsomething like this?
(defplugin no-attribute-errors-plugin
  {::pf.eql/wrap-map-select-entry
   (fn [original]
     (fn [env source ast]
       (when-not (= (:key ast) ::pcr/attribute-errors)
         (original env source ast))))})
#2021-07-2204:18markaddlemanI think you want https://pathom3.wsscode.com/docs/error-handling#lenient-mode
#2021-07-2205:33nivekuilnah, what I want is the old default behavior before the error handling shakeup, i.e. no strict and no attribute-errors-plugin. so far my code seems to be ok#2021-07-2300:22wilkerlucionot available at this moment, I wonder if that should be a thing (completely silent errors)#2021-07-2300:22wilkerluciocan you tell more about the reason you like to keep that?#2021-07-2300:25nivekuilspecifically, I make pathom calls to figure out what to put in a DB and that extra key was breaking things. in general it breaks an expectation I had that the output will always have a subset of the attributes of the query#2021-07-2300:27nivekuilI guess this is maybe a little un-clojurey where maps are supposed to be open, and you can assoc whatever keys and have consumers be agnostic to them, but it seems a lot harder to reach into say a nested entity after the query is done and recursively dissoc stuff than to just block it with a plugin#2021-07-2300:30nivekuilI also don't understand what I would ever do with the error keywords themselves, besides logging them, which would be done in a plugin#2021-07-2300:30nivekuilretries, different paths etc. seem like they'd just be done by pathom itself#2021-07-2300:49wilkerluciogotcha, I also had a few issues with Smart Maps and that behavior, so I can understand that we just want to keep strict at times, I may add another flag to disable the error thing#2021-07-2300:50wilkerluciomy reasoning to have that baked in as default is because of distributed Pathom calls, it could get really bad if the default was to hide errors, than would become a nightmare for distributed pathom debugging#2021-07-2222:59caleb.macdonaldblackI’m working with pathom3 and a relational database (PostgreSQL). 1. Would people recommend doing joins though EQL and pathom with batch resolvers, where resolvers return values from a single table? 2. Or should I join everything I need in single resolvers? Option 1 sounds awesome but will this work in the long run? I’m fairly new to Pathom#2021-07-2223:02markaddlemanIn my experience, the more work you can push down into the database (especially postgres), the happier you will be#2021-07-2223:04markaddlemanoption 1 would work but you'll be happier in a broader range of scenarios if you let postgres take on the bulk of the work#2021-07-2223:05caleb.macdonaldblackThank you @U2845S9KL, I appreciate your advice.#2021-07-2300:17wilkerluciohello calleb, I suggest you keep things simple as start, for db try to minimize the number of resolvers, so you minimize the number of round trips, so for example, to get the data from a user table, pull all the fields (but no relations) as a single resolver, use other resolvers for the relation traversing#2021-07-2300:18wilkerluciothere are ways you can optimise this in the future, you can ask pathom for example about which fields that resolver is supposed to cover (at running time, this is based on the user query and dependencies), so you can optimize the DB request to reduce the fields#2021-07-2300:19wilkerlucioin general, its not nescessary though, given for the DB the add of fields in the same row are usually very snappy, but I like to let you know it can get optimised if needed be (maybe there are 100 fields and most queries just cherry pick a few, in this case the optimisation makes more sense)#2021-07-2312:23souenzzoI think it's worth mentioning #walkable I don't think that you can directly use it because it is still in development but it has some nice ideas.#2021-07-2312:32souenzzohttps://github.com/walkable-server/walkable/#2021-07-2317:16wilkerlucioWalkable is great, but it isnt compatible with any other pathom stuff at the moment, it works only as standalone, that may change, I recently got progress enough on dynamic resolvers that we can start playing with that now#2021-07-2300:51wilkerlucioNew convenience helper p.eql/process-one landed on main! It allows for direct request for a single attribute, more info at https://pathom3.wsscode.com/docs/eql/#process-one#2021-07-2307:02wilkerlucioMinor release 2021.07.23-alpha includes the p.eql/process-one https://clojars.org/com.wsscode/pathom3#2021-07-2609:56cyppanIt would be nice to be able to rename fields in the EQL queries (like aliases in GraphQL)#2021-07-2611:33souenzzo@cyppan in pathom (2) there is an almost undocumented feature: [(:a {:pathom/as :b})] In pathom3 we still not have, as far I know#2021-07-2611:38souenzzobut it should be possible to implement in pathom3 as a plugin#2021-07-2611:39cyppangood idea, I’ll try that#2021-07-2611:44souenzzo@cyppan I think that is nice to have an "standalone" eql-select library, where you can simply call (select {:a 42} [(:a {:as :b})]) => {:b 42} Then develop a plugin to use eql-select as a pathom3 plugin#2021-07-2611:49cyppanThe Pathom2 way makes more sense to me#2021-07-2611:49cyppanJust to rename some fields on output#2021-07-2611:52souenzzoif anyone is interested in publishing an eql-select library here a "base implementation" from my codebase šŸ™‚ Maybe handle :default too on the library https://gist.github.com/souenzzo/cb37af999d2ab76e68e465427f1f3cd9#2021-07-2614:53wilkerlucioI'm trying to discourage this at this moment, but you can make an plugin using this extension point: https://pathom3.wsscode.com/docs/plugins#pcrwrap-merge-attribute#2021-07-2614:54wilkerlucioyou can read the params with (-> env ::pcp/node ::pcp/params), you can use that to re-implement something like :pathom/as#2021-07-2615:56cyppanAs a first very naive attempt, I used the wrap-map-select-entry extension point, following the protect-attributes-wrapper example
; define the extension wrapper fn
(defn rename-output-wrapper [mse]
  (fn [env source {:keys [key] :as ast}]
    (if-let [as (get-in ast [:params ::p.eql/as])]
      (coll/make-map-entry as (get source key))
      (mse env source ast))))

(p.plugin/defplugin rename-output-plugin
  {::pf.eql/wrap-map-select-entry rename-output-wrapper})

(comment
  (p.eql/process env {:todo/id 1} ['(:todo/name {::p.eql/as :todo/overriden-name})])
  )
#2021-07-2616:01wilkerluciolooks good, sorry I gave a bad instruction about how to get the params before, but you got it right šŸ‘#2021-07-2616:05wilkerluciobut there may be a problem there, I'm checking now, because going this way will prevent the children from use the same algorithm (it will not filter things that has :as)#2021-07-2616:05wilkerlucioyeah, confirmed the issue:
(p.eql/process
  (-> (pci/register [entity all-entities])
      (p.plugin/register rename-output-plugin))
  [{'(:all {::p.eql/as :foo})
    [:name
     ]}])
=> {:foo [{:id 1, :name "a", :address {:n 100}} {:id 2, :name "b"}]}
#2021-07-2616:05wilkerlucio(not asked for address, but ended up out there)#2021-07-2616:07wilkerluciofixed version:
(defn rename-output-wrapper [mse]
  (fn [env source ast]
    (let [entry (mse env source ast)]
      (if-let [as (get-in ast [:params ::p.eql/as])]
        (coll/make-map-entry as (val entry))
        entry))))
#2021-07-2616:08cyppanindeed better thanks !#2021-07-2616:15cyppanAnd for the why I need that, our front is in ReScript (so not very EQL-friendly), so I’m building a GraphQL server wrapper (using Lacinia) on top of Pathom resolvers on our backend API, and GraphQL supports that (to alias fields)#2021-07-2616:21souenzzo@cyppan if you are using lacinia, I think that it will do the alias for you, you don't need to worrry about that in pathom side.#2021-07-2616:37cyppanjust tried, you’re right šŸ‘#2021-07-2616:56souenzzoI did an (abandoned) prototype once Waiting to see your solution šŸ™‚#2021-07-2617:04wilkerluciothis is indeed something that I would love to see too, there is also a discussion for that: https://github.com/wilkerlucio/pathom3/discussions/18#2021-07-2706:31cyppanYeah not that easy I guess šŸ™‚ I’ll add a note on the discussion#2021-07-2716:12cyppanhttps://github.com/wilkerlucio/pathom3/discussions/18#discussioncomment-1063869#2021-07-2612:14cyppanOn the last alpha version (2021.07.23-alpha) I can see the pathom3 strict mode does not work for batch resolvers#2021-07-2614:49wilkerluciothis seems a bug, I can look at that today, thanks for the report#2021-07-2614:49wilkerluciobut I also wonder, why are you disabling cache on this resolver? cache is kinda important for Batch, because without reliying on the previous input to cache, its hard to know if that part was tried already.#2021-07-2614:59cyppanI guess I don’t really understand what is cached, if it is the resolver output, I don’t want caching because I often want the most recent data from the DB#2021-07-2615:14wilkerlucioby default the cache lives only during a single request#2021-07-2615:15wilkerlucioits to avoid re-doing the same operation from different parts of the same query#2021-07-2615:15wilkerluciohttps://pathom3.wsscode.com/docs/cache#cache-lifecycle#2021-07-2615:18cyppanAh! I missed that in the doc thank you > without reliying on the previous input to cache, its hard to know if that part was tried already. I don’t really understand that, I’m gonna have a look into the code#2021-07-2615:19wilkerlucioI gonna fix it anyway, because its a bug, it shouldn't rely only on that#2021-07-2615:20wilkerluciobut what happens, specially in the case of placeholders, is that you ahve to process the same stuff in different parts of the tree#2021-07-2615:20wilkerluciosince Pathom concept of entity is quite flexible, the only way to be sure we are trying to same thing is if the resolver gets called with the same exact input, from a different place#2021-07-2615:21wilkerluciothis when cache kicks in, also in the base of batch, the items are cached as if they were individual calls, so when we see that same data need in another part of the query, it can avoid re-calling the resolver by using batch#2021-07-2615:43cyppanI think I got it, thank you#2021-07-2612:27cyppanAnd I’m not sure how I should handle resources that do not exist in the database in my resolvers to avoid this :thinking_face:, throw an ā€œnot-foundā€ exception? It feels strange to me that the processor goes into an infinite loop (in ā€œlenientā€ mode) when it can’t get the expected output, what I would expect actually is a way to make the ::pco/output nilable, a bit like optional inputs (pco/? ...)#2021-07-2614:51wilkerlucioPathom 3 is still alpha, some things are still getting figured out, and batch is one of then, there's been some issues with the current way to process it, so its good to collect these kinds of issues so we can figure how to make a better implementation#2021-07-2614:51wilkerluciospecially with the very recent addition of strict mode, things are getting worked out, can you bring the specific case you have so we can see what we can do about it?#2021-07-2615:02cyppanNo problem with the alpha state, I’m happy to share and contribute in any way possible, Pathom is a really nice abstraction šŸ™#2021-07-2615:10cyppanHere I think my concern is not really the batch resolver: My backend is MongoDB so I have documents sometimes with keys sometimes without, I use those fetched documents directly in my Pathom resolvers, so it happens that a pco/output key is not present in the response. For instance, an account-resolver , for a :account/id input, declares the following outputs: :account/name and :account/avatar but avatars are optional, without even the avatar key sometimes in my Mongo documents, and this makes Pathom go into an infinite loop when I request :account/avatar in my EQL.#2021-07-2615:11cyppanI can merge a default map with all pco outputs keys with nil associated, but I wonder if there is something more elegant maybe to do#2021-07-2615:13cyppanlike an optional ::pco/output, but I’m not sure of the impacts on the execution plan#2021-07-2615:14cyppanIt’s more like a ā€œdefaultā€ for a ::pco/output maybe, but again, hard to integrate into the graph path resolution I guess#2021-07-2615:17wilkerlucioits ok to have optional things on the output, you can look at the output more as like: "I may provide these attributes"#2021-07-2615:18wilkerluciothe strictiness is always at the consumer side, so its not about the provider to say "I will for sure provide this", instead, its up to consumer to say "I *require* this part, while these other part may be optional"#2021-07-2615:18wilkerluciomakes sense?#2021-07-2615:21cyppanSince there is no concept of ā€œschemaā€, I guess the information of what keys are optional or not has to be expressed another way, so yes it makes sense#2021-07-2615:21cyppangiving that responsibility to the consumer#2021-07-2615:22wilkerlucioyeah, and I think this a good place to be, in the same way we want the consumer to decide what he wants in the query, he is also responsible for deciding what is optional about it#2021-07-2615:22wilkerlucioand to point out, when I say consumers I'm refering to resolver inputs and queries sent to p.eql/process#2021-07-2615:23cyppan(I was thinking of several resolvers* providing the same outputs, and how to choose / fallback to another path when you have required and optional outputs, wondering how to handle that)#2021-07-2615:26cyppanwhen you say ā€œits ok to have optional things on the outputā€ you mean, as a resolver, I’m not forced to provide you the keys I have declared?#2021-07-2615:26wilkerlucioon multiple options Pathom will generate OR nodes in the plan, so it will try one at a time, there is a built-in thing you can use to prioritize, but this is still an area of exploration, not many people have used it so not enough experience, but there is a discussion about it here: https://github.com/wilkerlucio/pathom3/discussions/57#2021-07-2615:27wilkerlucio> when you say ā€œits ok to have optional things on the outputā€ you mean, as a resolver, I’m not forced to provide you the keys I have declared? correct#2021-07-2615:27cyppanOk, when the bug I’ve pointed out is solved it should work directly cool#2021-07-2615:27wilkerlucioPathom never complains about missing data on the output, it complains if the the data is missing by the time it tries to call something that requires it#2021-07-2615:29wilkerluciohere is an example that illustrates that:
(pco/defresolver ab []
  {::pco/output
   [:a :b]}
  {:a 1})

(pco/defresolver c [{:keys [a]}]
  {:c (inc a)})

(p.eql/process-one
  (pci/register [ab c])
  :c)
#2021-07-2615:29cyppanYes I understand, this is problematic in my case because I have ā€œforeign keysā€ in my mongo documents, but sometimes without the key as I was explaining, so when I try to ā€œjoinā€ and there isn’t the key, it errors, I’m merging nil keys to solve that#2021-07-2615:30wilkerlucioab says it outputs :b, but it doesn't, but since :c doesn't care about :b. not an issue#2021-07-2615:30wilkerlucioits valid to make join keys optional, is this something that works for your case?#2021-07-2615:30wilkerluciolike: [:foo {(pco/? :join-bar) [:baz]}]#2021-07-2615:34cyppanOh! what is this, a resolver output ?#2021-07-2615:35cyppan#2021-07-2615:36wilkerluciono, on the input (or query)#2021-07-2615:36wilkerlucioresolver outputs never need optional mark#2021-07-2615:38wilkerluciocan you show the place where you demand the join? I know its a bit weird at first this thing about requirements on the consumer#2021-07-2615:39cyppanSorry, not sure I follow you#2021-07-2615:39cyppanin my case I do something like that, [:meta-campaign/brand {(pco/? :brand_expanded) [,,,]]#2021-07-2615:39cyppanthe join key is :meta-campaign/brand, which sometimes is not present from my meta-campaign resolver output#2021-07-2615:40cyppanIf I understand correctly it should work with [:meta-campaign/brand {(pco/? :brand_expanded) [,,,]] but not with [:meta-campaign/brand {:brand_expanded [,,,]]#2021-07-2615:41wilkerluciolet me coin an example of what I have in mind, so we can go around it#2021-07-2615:43wilkerlucio
(def entity-db
  {1 {:name    "a"
      ; has address
      :address {:n 100}}
   2 {:name "b"
      ; no address
      }})

(pco/defresolver entity [{:keys [id]}]
  {::pco/output
   [:name {:address [:n]}]}
  (get entity-db id))

(pco/defresolver all-entities []
  ;just output the ids
  {:all
   (mapv #(array-map :id %) (keys entity-db))})

(p.eql/process
  (pci/register [entity all-entities])
  [{:all [:name {(pco/? :address) [:n]}]}])
#2021-07-2615:43wilkerlucionote the optionality on the query side, and this works the same when making ::pco/input#2021-07-2615:45cyppanI see, I understand better what you meant by ā€œrequirements on the consumer sideā€#2021-07-2615:46cyppanit would be great to have this example in the doc#2021-07-2615:46cyppanthank you for your time šŸ™‚#2021-07-2615:47wilkerluciono worries, glad we got thought it#2021-07-2615:47wilkerlucioif you find a good place to add this example, PR is welcome šŸ™‚#2021-07-2615:47wilkerluciomaybe on the optional input section#2021-07-2615:48cyppanyep was thinking about there#2021-07-2612:42souenzzoit seems a pathom3 bug#2021-07-2714:48wilkerlucio#2021-07-2714:48wilkerlucio@U0CL38MU1 includes the fix for that batch cache case#2021-07-2714:52cyppanšŸ’Æ thanks#2021-07-2715:13markaddlemanCongrats on the release! I don't think ::pco/transform is documented in the Pathom 3 docs. At least, I couldn't find it with a simple search#2021-07-2715:14wilkerluciowell noted, it really isn't, adding to the todolist šŸ™‚#2021-07-2715:16markaddlemanNo rest... šŸ™‚ Thanks!#2021-07-2715:53wilkerluciodeploying https://github.com/wilkerlucio/pathom3-docs/commit/abc933a3d141277c08a476dc51d2ca15de1c5118#2021-07-2715:57wilkerlucioits out! https://pathom3.wsscode.com/docs/resolvers/#resolver-transform#2021-07-2718:12markaddlemanOh! This is awesome and I think I have a use case for it in my current project#2021-07-2720:24wilkerluciothe example was wrong, publishing a fix now#2021-07-2715:57wilkerlucioits out! https://pathom3.wsscode.com/docs/resolvers/#resolver-transform#2021-07-2716:39genekimI’m trying to puzzle out why an ā€œN+1ā€ query is happening, despite writing a batch resolver (holy cow, I just saw how batch-transforms now makes it easier to write batch code. Nice!!! I can’t wait to get rid of all those (if (sequential? input)) from code! šŸ™‚ I have the following resolvers to 1) fetch list of all Trello boards, and 2) fetch all the lists associated with each board.
(pc/defresolver trello-boards [env input]
  {;::pc/input  #{:trello/board-id}
   ::pc/output [{:trello/boards
                 [:trello-board/id :trello-board/name :trello-board/starred
                  :trello-board/lists]}]})

(pc/defresolver trello-board-lists [env input]
  {::pc/input  #{:trello-board/id}
   ::pc/output [:trello-board/id
                {:trello-board/lists
                 [:trello-list/id :trello-list/name :trello-list/board-id
                  :trello-list/pos]}]}
  ::pc/batch? true})
When I execute this query, the trello-board-lists resolver is only called with maps, never sequentials, so the lists are always fetched sequentially, not in parallel. Does anyone have any advice on how I can make sure that resolver is called in a way that always triggers batching? Thank you!
[{:trello/boards
      [:trello-board/id :trello-board/name :trello-board/lists]}]
#2021-07-2716:44genekimAh, I meant to say more explicitly ā€œthank you, @U066U8JQJā€ for the batch-transform feature. Lovely!#2021-07-2716:45wilkerluciothanks @U6VPZS1EK šŸ™‚. this was also simplified in Pathom 3, there batch resolvers always get a sequence (never a single entry)#2021-07-2716:47wilkerlucionot sure if I understand your case in full, but in Pathom 2 there is a limitation in batch, it only works when the items are an immediate sequence, (like, the item directly under some collection)#2021-07-2716:47wilkerlucioif there is a more than 1 level separating the list from the batch attribute, it doesn't work#2021-07-2716:47wilkerluciothis was also fixed in Pathom 3, where it doesn't care at all about this kind of structure, and can batch across any different points of the query#2021-07-2716:51genekimThank you! A wonderfully simple explanation! (I’ll rewrite the query. But now I’m curious about what @U0CKQ19AQ’s plans are on whether / when Pathom 3 migration may happen.) Onwards! šŸ™‚#2021-07-2716:52wilkerluciofor Fulcro users it should be a simple change in terms of swiping one from the other, for Fulcro all that matters is a EQL engine to fulfill the requests. the work will be migrating the current implementation on the server to use Pathom 3 things, but on the client the changes should be trivial#2021-07-2716:53wilkerlucio(I already have a couple of toy projects using Fulcro and Pathom 3 that I do for experimentation)#2021-07-2808:35cyppanIn lenient mode, when I query a field with (pco/? :field) and the field is not returned by the resolver, I get an attribute error, but I would expect no error, the field should simply not be in the output (or do I miss something?)#2021-07-2816:11wilkerluciohttps://github.com/wilkerlucio/pathom3/issues/80#2021-07-2816:36cyppanDo you have any doc or advice to start contributing ?#2021-07-2817:20wilkerluciono docs yet,but would be nice to have#2021-07-2817:21wilkerlucioabout this specific issue, I fixed already, but I left home and forgot to push :face_palm:, will push once I get back home in a few hours#2021-07-2817:21wilkerluciobut I suggest you can start familiarizing with the namespaces and how the concepts are organized#2021-07-2817:22wilkerlucioIm happy to go over any questiona you have on it#2021-07-2817:40cyppanok thanks I’ll follow your suggestion#2021-07-2821:08wilkerluciopushed#2021-07-3014:45cyppanNow I don’t have the errors when the attributes are declared in the resolver pco/output, but if they’re not, an EQL query with a (pco/? :undeclared-output-key) will add an attribute error :com.wsscode.pathom3.error/attribute-unreachable It forces me to declare all the possible output keys in my resolvers, which I wanted to avoid, is it a wanted behavior?#2021-07-3014:52cyppanWhen I think about it the error is legit, what I would want actually is to declare the strict minimum in my resolvers, only some keys and what’s necessary for joins, but the rest would be fed from the resolver data when possible, a kind of a new kind pco/?, maybe a pco/try or something like that#2021-07-3014:56cyppanjust seen your issue https://github.com/wilkerlucio/pathom3/issues/81#2021-07-3014:58cyppan> In lenient mode, an optional attribute missing on the index adds an error, it shouldn’t by ā€œmissing on the indexā€ you mean it’s not available in a reachable pco/output right ? (Oh no this corresponds to the one just after it, I understand it’s in the case where there is no pco/output at all corresponding to this in any resolver in the env)#2021-07-3016:14wilkerluciomissing on the index means there is no resolver that contains that attribute at the resolver output root level (nested levels have a different process)#2021-07-3016:15wilkerlucioI plan to tackle this issue today#2021-07-2822:45nivekuilin strict mode, when I request only an optional attribute, pathom will still throw. is that expected?#2021-07-2823:31wilkerlucioexample? and consider the comment I sent on the other message :)#2021-07-2822:48nivekuilin general it seems if there isn't a path to an attribute it doesn't matter whether it's a pco/? or not#2021-07-2823:30wilkerlucioin strict mode, exceptions will always flows up#2021-07-2823:30wilkerluciothe exception is if the attribute has multiple options and only some of them fail, in this case it doesnt throw#2021-07-2823:32wilkerlucioah, yes, thats correct, in strict mode if you ask something not on the index, always a failure#2021-07-2823:45wilkerluciobut this is not set in stone, thinking a bit more about it, maybe makes more sense to ignore that optional in case of absense#2021-07-2823:46wilkerlucioI'm thinking that it should behave the same way in inputs and queries, and in inputs its not an exceptional thing if the attribute is not in the index at all (for optionals)#2021-07-2903:35nivekuilat least my intuitive expectation is that if a requested attribute is optional, it should have no effect at all on the rest of the query, and all the processing steps done solely for its sake should be considered lenient, but I can see how that adds a lot of complexity to pathom#2021-07-2903:35nivekuilnot being able to find an optional attribute also adds an attribute error to lenient mode btw#2021-07-2904:38wilkerluciothanks for the input, I opened an issue, should address them soon: https://github.com/wilkerlucio/pathom3/issues/81#2021-07-3018:26wilkerlucioPathom 3 2021.07.30-alpha just released! This makes adjusts to error handling in optional items in Lenient mode, except for error cases, optionals will ignore problems of attribute not existing (not defined by any resolver) or unreachables (attributes that don't have enough data to get into) https://clojars.org/com.wsscode/pathom3#2021-07-3107:52weiwhen writing pathom resolvers for datomic, is it better to key on the primary key (e.g. uuid) or the internal db/id ? what if the entity has a composite primary key, is it ok to key off that?#2021-07-3114:57markaddlemanI'm curious what others have to say. In my experience, I prefer not to expose internal IDs so I would avoid the datomic I'd and use the generated uuid#2021-07-3114:59markaddlemanComposite keys are interesting. You can certainly model them using Pathom. There is some question around how to expose that data in external facing Pathom API#2021-07-3114:59markaddlemanSee https://github.com/wilkerlucio/pathom3/discussions/77#discussioncomment-1048391#2021-08-0207:04weigreat, thanks for the feedback. think i'll go with uuid.#2021-08-0516:32eoliphant+1 for domain/biz/etc id’s instead of :db/id. Pathom and Datomic are a match made in heaven, EQL all the way down (more or less), but :db/id’s have always felt a little leaky to me#2021-07-3117:35lgesslerIs there a way to ask pathom to give me "everything" that can be reached via joins on a certain entity? E.g, if I have an entity type a with :a/b as a join, and b with b/c as a join, I'd want results like {[:a/id ...] {:a/id ... :a/b {:b/id ... :b/c {...}}}} given a query like [{[:a/id ...] ???}] . I feel silly asking, since I know pathom's all about specific queries yielding specific data, but I'm feeling lazy#2021-07-3117:44lgesslerI know the * special query symbol is available for a particular entity, i guess i'm looking for a recursive version of that#2021-07-3118:40lilactownyou could try:
[{[:a/id 1] [* {:a/b [* {:b/c [*]}]}]}]
#2021-07-3118:41lilactownI think fundamentally pathom will not traverse joins unless explicitly asked#2021-07-3119:22lgessleryeah... the problem here is that I have joins of unknown depth/type. No biggie, I should just write some code šŸ™‚#2021-08-0218:26wilkerluciono feature for that yet, in Pathom 3, a good way to explore data you don't know is using smart maps + datafy#2021-08-0218:26wilkerlucioif you do this combo, you can use tools like Portal, Reveal or REBL to navigate the structure#2021-08-0218:26wilkerluciois your usage for exploration or to make something prod-like?#2021-08-0220:51lgesslerprod-like: there's a tree I want to emit that has an unknown (user-created) structure. I think I wanted this facility to help me tinker with stuff in the REPL while I'm developing without writing a giant EQL query, but for actual prod code I don't think this is something I need, and for dev I can just write a function in my dev ns to help with the query#2021-07-3122:51nivekuil@wilkerlucio is https://github.com/wilkerlucio/log meant for public use? going to polish up and release my pathom-based depenedncy injection framework, think it's pretty good at this point and was wondering if I can make use of this#2021-08-0218:25wilkerluciohello, feel free to use, I'm not decided if I'm going to continue or not this library, but so far I see no other option for map-driven logs that works on both CLJ and CLJS, would love to use mulog, but it doesn't work in CLJS#2021-08-0116:24royalaid@wilkerlucio just wondering if there are any updates on possible pathom3 parallel parser? We are just waiting for that before migrating from 2 to 3#2021-08-0218:25wilkerluciohello, parallel is the next phase of Pathom 3, currently I'm finishing things up on dynamic resolvers, once this is ready and documented I'll move the focus to parallel processing#2021-08-0220:54JonRHey channel, wondering if anyone has experience embedding pathom-viz in a cljs app? I think I'm close but having a couple issues following the docs#2021-08-0220:55JonRThe #:pathom.viz.embed reader syntx is giving me issues. I'm also not clear on how to construct this data structure specific to my data model/app#2021-08-0220:56JonR@wilkerlucio ^. Thanks for any info!#2021-08-0317:05wilkerluciohello @jon693, currently integrating the full pathom viz isn't a trivial task, but if you wanna give a go you can use this file as a base (this is how its integrated in the Electron app): https://github.com/wilkerlucio/pathom-viz/blob/master/src/electron/com/wsscode/pathom/viz/electron/renderer/main.cljs#2021-08-0317:05wilkerlucioI hope to make this a simpler task in the future, but a lot of refactoring is required for that#2021-08-0317:06wilkerluciothat said, there are a few parts of Pathom Viz that are simple to integrate, those are: - the trace view - the node graph for those you can use Pathom Viz embed, which is something you integrate kinda like you would put a youtube video embed in a page#2021-08-0317:07wilkerluciothere is some example code to wrap those as react components at the pathom 3 docs source: https://github.com/wilkerlucio/pathom3-docs/blob/master/src/main/com/wsscode/pathom3/docs/components.cljs#2021-08-0317:08wilkerlucioand docs here: https://github.com/wilkerlucio/pathom-viz#embed#2021-08-0322:06JonRthanks @U066U8JQJ, this is just the info I needed. I think it's a bit larger of an undertaking to take on at the moment then I was hoping. I'll use the electron app for now.#2021-08-0322:06JonRLoving this workflow and the pathom-viz tool though FYI. Awesome work#2021-08-0417:22JAtkinsIs this the current pattern for pathom viz connecting with pathom3?
(def parser
      (cond->> (p/parser ...)
        dev-mode?
        (p.connector/connect-parser
          {::p.connector/parser-id ::my-parser})))
It’s in the docstring of com.wsscode.pathom.viz.ws-connector.pathom3/connect-env . I’m slightly confused seeing the two functions refer to eachother.
#2021-08-0418:14wilkerluciopathom 3 uses a different namespace from the connector#2021-08-0418:14wilkerluciobut the naming still on the other old one (the attribute with id)#2021-08-0418:15wilkerluciobut there is a shortcut arity to avoid the long name#2021-08-0501:00dehliIs it possible to configure a resolver in pathom2 so it never stores its results in request-cache ?#2021-08-0513:54wilkerlucio::pc/cache? false on the resolver configuration#2021-08-0514:21dehliawesome! thanks šŸŽ‰#2021-08-0515:06cyppanStarted the migration from a custom store api (based on https://github.com/cyppan/grape) to Pathom 3, with batching and async, and look at the number of opened connections to Mongo already... šŸ˜„#2021-08-0608:31Pragyan TripathiWow… this makes me sure I should try it out in my own project..#2021-08-0520:37nivekuilbug report: process-one will NPE if the attribute it gets back is nil, because of val#2021-08-0613:44wilkerluciocan't reproduce, do you have an example?#2021-08-0620:54nivekuilwell, it happens if process returns {} so it tries to call (val (first {}))#2021-08-0620:56nivekuilso all you have to do is ask for attribute that isn't in the index (with lenient on)#2021-08-0621:32wilkerlucioI tried an example but I didnt see the error, a broken example really helps so I dont have to assume anything (like lenient mode)#2021-08-0700:50nivekuilhuh, that's harder to repro than I thought. I figured it would be an easy fix since process can return empty map (according to its spec too), in which case you can see that it will always NPE#2021-08-0700:51nivekuile.g. (peek (first instead of (val (first#2021-08-0822:45plinsIm trying to learn pathom by building a simple CRUD api, and I wonder what is the pattern (if there is one) on what is returned on mutations? should I return on update/create?#2021-08-0823:51wilkerluciousually you can return the map of the just created entity, so the user can read from it if desired#2021-08-0900:33souenzzoConsider a schema where a "user" has a "cart" When the user call (create-cart {}), I would return
(pc/defmutation create-card ....
   {:user/id ...
    :cart/id ...})
Then, the frontend can call both: [{(create-cart {}) [:user/name {:user/cart [:cart/id :cart/empty?]}]}] [{(create-cart {}) [:cart/id :cart/empty?]}]
#2021-08-0900:33souenzzoIMHO, one of the main points of pathom/graph API's is that you don't need to think about "what to return"#2021-08-1117:03lgesslerI want to ensure that all my mutations run "synchronously" (one-at-a-time, to completion, across all threads) so I won't have to worry about some race conditions--does anybody have a recommended way of doing this? perhaps by wrapping my call to (parser env tx) with some kind of mutex if tx is a mutation?#2021-08-1117:21souenzzo@lgessler you can do something like that:
(def single-parser
  (let [in (async/chan)]
    (async/thread
      (loop []
        (when-let [{:keys [env tx out]} (async/<!! in)]
          (async/>!! out (parser env tx))
          (recur))))
    in))

(comment 
  (defn handler 
    [req]
    (let [out (async/chan)]
      (async/>!! single-parser {:env env :tx tx :out out})
      (async/!! out))))
Or try to use "wrap-mutation-plugin" to make it serial, but only if it is a mutation call But why mutation races are a problem? All that solutions will only work in a single machine, they do not scale
#2021-08-1117:22lgesslerthanks, that looks like a good solution! yeah, I'm OK with all the downsides (single-node, low write throughput) for now--it's just more important that I get this correct and shipped atm#2021-08-1200:19caleb.macdonaldblackWhy does this happen in Pathom3?
(pco/defresolver foobar []
  {::pco/output [:id :name]}
  {:id 1 :name "foobar"})

((p.eql/boundary-interface
   (pci/register [foobar]))
 {:pathom/eql [:id :name] :pathom/entity {:id 0}})

=> {:id 0, :name "foobar"}
It feels odd that that id is my input by my output is also included.
#2021-08-1200:20caleb.macdonaldblackI was writing a test and was not expecting this behaviour. I feel like it could introduce bugs where the wrong resource is returned but it looks good because the id matches now#2021-08-1201:05wilkerlucio@caleb.macdonaldblack this happens because your resolver doesn't require any input, so this makes :id and :name available globaly#2021-08-1201:06wilkerlucioso what pathom sees is that you have already an id (wiht value 0), and the missing name sees a resolver that requires no input, so it pulls from that, makes sense?#2021-08-1201:06wilkerlucio(it also works the same in Pathom 2)#2021-08-1201:07caleb.macdonaldblackI removed the input to simplify but it’s the same behaviour with an input#2021-08-1201:07caleb.macdonaldblack
(pco/defresolver foobar [{:keys [id]}]
  {::pco/output [:id :name]}
  {:id 1 :name "foobar"})

((p.eql/boundary-interface
   (pci/register [foobar]))
 {:pathom/eql [:id :name] :pathom/entity {:id 0}})

=> {:id 0, :name "foobar"}
#2021-08-1201:08wilkerluciosame thing, in this case, :name is missing, so it uses the ID, given the output is fixed, it fills the missing parts#2021-08-1201:08wilkerluciobut pathom wont ever override values already present#2021-08-1201:08wilkerlucioso you had :id 0 from start, Pathom will never change the data, that's why you see what looks inconsistent, but in this case I think we can argue that the resolver is returning something inconsistent based on the id#2021-08-1201:11caleb.macdonaldblackYeah I agree the resolver is bad. If there was a bug with the resolver where the input isn’t used, it could be combined with irrelevant output and wouldn’t be obvious. This happened when I was writing a test#2021-08-1201:11caleb.macdonaldblackIt’s nothing major, I was more curious if anything#2021-08-1201:11wilkerluciosure, no problem, its good to clarify the intentions šŸ‘#2021-08-1201:12wilkerluciobut I hope the process makes sense, if Pathom did override data it would be madness šŸ˜›#2021-08-1201:12caleb.macdonaldblackActually that does make sense now#2021-08-1201:12caleb.macdonaldblackImagine that haha#2021-08-1201:13wilkerluciothere is one idea though, that I think would be useful as a developer tool, is some plugins to validate output#2021-08-1201:13wilkerlucioso in a case like this, pathom could see a resolver trying to modify some already present data, anda this is a strong indication that something is flawed in the logic of the resolvers#2021-08-1201:14wilkerlucioand notify the user in some way, or even break#2021-08-1201:14caleb.macdonaldblackAh yeah, that would’ve been a good indicator for me. I can’t imagine this happens often though.#2021-08-1201:16wilkerlucioyeah, the idea is to be some kind of linter thing, another thing I would like to catch for example, is if the output contains more keys than expressed in ::pco/output, this is a strong indicator that the output definition is incomplete#2021-08-1205:39cyppanWe use that a lot (incomplete output declarations), only putting needed keys for pathom to resolve the graph (like joins), to me it’s actually a feature, at least for our usage. We do that because it allows our (quite rich and complex) data to evolve in an independent way without impacting every time the resolver layer.#2021-08-1205:41cyppanBut yeah an optional « lintĀ Ā» plugin looks like a good idea, for more strict use cases.#2021-08-1216:59wilkerlucioyeah, the motivation for that is to avoid situations where an attribute works on a query, but not in another, for example, if a resolver declaration says it only outputs :a, but the code outputs :a and :b, a query for [:a :b] will get both values, but a query for [:b] will get nothing#2021-08-1217:00wilkerlucioits a subtle issue that can get painful to find out#2021-08-1217:00wilkerluciobut I'm realizing I have this in my head for a long time, and it isn't a problem on strict mode (that would reject the request for :b given it doesnt know about it)#2021-08-1217:00wilkerluciobut that's still a thing for lenient mode#2021-08-1217:55cyppanI understand, I’m personally fine with this behavior#2021-08-1217:58wilkerluciocool, and it will not change šŸ™‚#2021-08-1201:18wilkerlucionot my priority to make this, but if anything would want to give it a try, the code could be later integrated as a built-in plugin#2021-08-1218:42markaddlemanI'm starting to use caching in Pathom3. The API is very easy to integrate, so thanks! One oddity: From what I understand, resolvers require caches to support protocol pcache/CacheStore but the plan cache does not. The plan cache assumes a cache atom from clojure.core.cache . Does it make sense for the plan cache to support the protocol? I think that would mildly simplify the API#2021-08-1218:58wilkerlucio@markaddleman core.cached is a separate library, and it doesn't work on CLJS for example, so I dont plan to integrate directly with that (altough there are code examples on how to integrate with it)#2021-08-1218:58wilkerlucioPathom extends atoms and volatiles to implement pcache/CacheStore protocol, and that's whats used by default everywhere#2021-08-1219:00markaddlemanIn the latest alpha, I think Pathom specs require that the plan cache be an atom. I supplied a pcache/CacheStore record to it and Pathom got very upset#2021-08-1219:00wilkerluciorepro demo?#2021-08-1219:00markaddlemanSure, one sec#2021-08-1219:07markaddlemanPathom with guardrails enabled reports a lot of spec failures#2021-08-1219:40wilkerluciooh, right, the specs, yeah, I can see the spec is wrong, I can fix it later today, thanks for the report#2021-08-1219:40wilkerlucio@markaddleman do you mind opening an issue just for tracking?#2021-08-1219:40markaddlemanno problem#2021-08-1219:42markaddlemanhttps://github.com/wilkerlucio/pathom3/issues/88#2021-08-1219:43wilkerluciothanks šŸ™#2021-08-1221:18wilkerlucioupdate landed on master, can you test if works as expected in your case now?#2021-08-1223:51markaddlemanWill do#2021-08-1318:59markaddlemanFinally got a chance to test the commit. Looks good#2021-08-1319:00wilkerluciothanks, I'll cut a new release with that later today#2021-08-1421:21wilkerlucio#2021-08-1612:04donavanIs P3 able to distinguish based on nested outputs? Say I have the following two resolvers:
(pco/defresolver something-else
  [_ _]
  {::pco/output [{:container [:something-else/id]}]}
  {:container [{:something-else/id :bar}]})

(pco/defresolver something
  [_ _]
  {::pco/output [{:container [:something/id]}]}
  {:container [{:something/id :foo}]})
can I then disambiguate with a nested query like:
(p.eql/process  
 env2
 {}
 [{:container [:something/id]}])
From basic testing in the repl it doesn’t seem so… the resolver last in the pci/register call wins, so is this something P3 cannot do?
#2021-08-1619:30wilkerluciohello Donavan, I have plans to support it, good to know about someone trying to use it, I kinda started but didn't finished that part yet: https://github.com/wilkerlucio/pathom3/blob/master/src/main/com/wsscode/pathom3/connect/planner.cljc#L926-L928#2021-08-1619:57donavanAwesome šŸ˜„#2021-08-2020:50kendall.buchanan+1#2021-08-3114:59donavan@wilkerlucio I took a stab at implementing this… would it best be done in the initial planning phase or as in the optimisation phase?#2021-08-3115:00wilkerlucioI add the comment in the part of code that needs to change, thats inside that function#2021-08-3115:01donavanmmmm, that function is never called when I try run the example above#2021-08-3115:02donavan…but that answers the question, it’s during initial planning not optimisation which is what I would expect#2021-08-3115:07wilkerluciommm, there is another problem to this situation that occurred to me#2021-08-3115:08wilkerluciothe nested checks are only done for dynamic resolvers, not for static ones#2021-08-3115:10wilkerluciothat's something to consider, if the planner should try going on nested things always, have to think about how that could impact performance#2021-08-1619:29royalaid@wilkerlucio I am working through integrating guardrails into our in-house pathom2 macros. What would be the best way to go about supporting it? My current through it pull out the gspec parsing and run-check functions in guardrails and use those in the forms emitted by the macros#2021-08-1619:30royalaidIdeally this will look like an addition gspec line in the pathom mutations/resolvers#2021-08-1619:31wilkerlucio@royalaid if you add specs to a macro, they will always run automatically by Clojure#2021-08-1619:48royalaidRight but I want to add gspec style arg checking#2021-08-1619:48royalaidI am afk atm but will sketch out an example once I am back at my keyboard#2021-08-1621:00royalaid
(pc/defresolver ex-one
  "Example Resolver"
  [{:keys [foo}]
  {:stack/code-url (when foo
                     (str "" foo))})
#2021-08-1621:01royalaidSo for the above I imagine something like#2021-08-1621:01royalaid
(pc/defresolver ex-one
  "Example Resolver"
  [{:keys [foo}]
  [map? => map?]
  {:stack/code-url (when foo
                     (str "" foo))})
#2021-08-1621:02royalaidFor the sake of example, this is a terrible spec but it illustrates what I am trying for#2021-08-1621:27wilkerlucioyou should consider if its worth to do it, resolvers will always be [map, map] => map (exception on batch, which is [map (coll map)] => (coll map), so what is the gain to add it? another idea is to instead use a plugin, with a plugin you can read input/output descriptions strait from the resolver, and than you can run spec checks on top of that#2021-08-1921:59Drew Verleegreetings, i'm reading the wonderful written documentation for pathom3 and evaling some examples in my editor and i seem to have hit a snag. https://pathom3.wsscode.com/docs/resolvers/#parameters talks about (pf.eql/process env [::todos]) And i'm told that alias is this: [com.wsscode.pathom3.format.eql :as pf.eql] However i can't seem to find that function. I'm on the latest commit 6e8fa2#2021-08-1922:00Drew Verleeis it supposed to be one of the other eql namespaces?#2021-08-1922:01Drew Verleejust p.eql/process maybe?#2021-08-1922:02Drew Verleethat seems to do the trick.#2021-08-1922:03Drew Verleeif someone can point out where i should submit documentation changes ill be happy to note it there šŸ™‚#2021-08-1923:35wilkerluciosorry, I see that too late, but at the end of all documentation pages there is a link to "Edit this page", you can use that to send edits#2021-08-1923:35wilkerluciofix commit for the pf.eql: https://github.com/wilkerlucio/pathom3-docs/commit/c7b694b29a5739884ce1300993a0acb6d7bfcb15#2021-08-1923:33wilkerluciohello, @drewverlee ups, that example is wrong, it should be p.eql/process, from com.wsscode.pathom3.interfaces.eql#2021-08-1923:33wilkerluciogoing to fix that now#2021-08-1923:57Drew VerleeSo this mock example works the way i would hope
(def mock-todos-db
  [{::todo-message "Write demo on params"
    ::todo-done?   true}
   {::todo-message "Pathom in Rust"
    ::todo-done?   false}])

(pco/defresolver todos-resolver [env _]
  {::pco/output
   [{::todos
     [::todo-message
      ::todo-done?]}]}

  {::todos
   ;; pull params and filter
   (if-some [letter (get (pco/params env) ::letter)]
     (->> mock-todos-db
          (filter #(str/includes? (::todo-message %) letter)))
     mock-todos-db)})


(def env (pci/register todos-resolver))

(def sm2 (psm/smart-map env {}))


(p.eql/process env
               '[(::todos {::letter "x"})])
But it doesn't work for a more complex example because I don't now how to structure the eql correctly I suppose. Some simple questions: 1. where would i read more about the syntax for [(::todos {::letter "x"})] E.g i assume ::todos patches to the defresolver top level map key, this is EQL? 2. what does the :> syntax mean? 3. Is pco output always a vector? Is that to allow for multiple root/maps? I'm reading the docs know. I'll try to answer my own simple questions as I have time ;).
#2021-08-2002:50wilkerluciono worries 1. you can read about the syntax at: https://edn-query-language.org/eql/1.0.0/specification.html#_parameters 2. Placeholders: https://pathom3.wsscode.com/docs/placeholders 3. yes, always a vector, a EQL more specifically, and the resolver output is always a map (*both things change in case of batch resolvers)#2021-08-2002:50wilkerluciocan you give more details about the more complex example you are trying?#2021-08-2002:52wilkerluciops: on your demo, better use filterv, lazy sequences may cause strange trace behavior (due due to when the item collections get evaluated)#2021-08-2005:15JAtkinsCan nested entities ā€œseeā€ the outer context while in a pathom process? I thought I saw this in use at some point, but it’s not working now.
(p.eql/process
  {}
  {:root/some-data :some-data
   :nest {}}
  [:root/some-data
   {:nest [:root/some-data]}])

=> {:root/some-data :some-data, :nest {}}
#2021-08-2011:55wilkerluciono, they cant, if you want that the parent must forward that data down#2021-08-2013:00JAtkinsHow could I make the data forwarded?#2021-08-2013:10wilkerluciocopy it over at the higher resolver#2021-08-2013:11wilkerlucioin the resolver of :root/some-data, get the input and copy into the items, makes sense?#2021-08-2013:13JAtkinsoh, gacha#2021-08-2015:36Drew Verlee
;; I'm not sure how to extend this resolver to use a pathom parameter

(pco/defresolver foo
  [{id :a/id}]
  {::pco/output [:a/b :a/c]}
  (get {1 {:a/b 1 :a/c 2}} id))

;; first, it seems necessary to extend the number of arguments it takes to get the env variable

(pco/defresolver foo1
  [env {id :a/id}]
  {::pco/output [:a/b :a/c]}
  (get {1 {:a/b 1 :a/c 2}} id))

;; as an aside its odd that even though the first argument to p.eql/process is the id, it now comes as the second argument

(p.eql/process (pci/register foo1)
               {:a/id 1})

;; The real issue how ever is how to pass a pathom parameter now

(p.eql/process (pci/register foo1)
               {:a/id 1}
               ;; i'm assuming this should be of the grammar here 
               )

;; But it's not clear how or what. First lets use our param instead of the input


(pco/defresolver foo2
  [env {id :a/id}]
  {::pco/output [:a/b :a/c]}
  (get {1 {:a/b 1 :a/c 2}} (-> env pco/params :param/id)))


(p.eql/process (pci/register foo2)
               {:a/id 1}
               );; => nil

(p.eql/process (pci/register foo2)
               {:a/id 1} ;; ignoring this on purpose
               ;; if we look at this example 
               ;; (p.eql/process env ['(::todos {::todo-done? false})])
               ;; then we should pass a vector, but the second argument is a list with one key. What does that mean?
               ;; in our example here, the output isn't one map but two, so where does the "with param"/param go?
               ;; we trying something random
               ['(:a/b {:param/id 1})]
               ;; it doesn't work
               );; => nil
#2021-08-2019:17wilkerluciohello, can you tell a bit more concretely what you looking for? I'm not sure if I get what you trying to do yet#2021-08-2019:19wilkerlucioabout the signature, think this way, resolvers *always* take 2 parameters, the thing is that the defresolver macro will fill in if env is missing, but if you need env (and you do need it for params), then you add it before#2021-08-2019:20wilkerlucioI think the following example may help with it:#2021-08-2019:20wilkerlucio
(pco/defresolver param-item [env {:keys [id]}]
  {::pco/output [:a :b]}
  {:a (pco/params env)
   :b (pco/params env)})

(comment
  (p.eql/process
    (pci/register param-item)
    {:id 1}
    ['(:a {:foo "bar"})
     ':b]))
#2021-08-2019:20wilkerluciooutputs:
=> {:a {:foo "bar"}, :b {:foo "bar"}}
#2021-08-2019:20wilkerluciowhen doing params with resolvers that have multiple outputs, remember they will share the params#2021-08-2019:21wilkerluciowould love some feedback on how we can improve the docs for it#2021-08-2117:27Drew Verlee@U066U8JQJ i'm mostly trying to learn pathom, so this is very helpful. Interestingly, the code i linked actually now returns the correct result. Which makes me believe my repl had ended up in some corrupted state. Such is life? I'm hesitant to recommend changes to docs without a better understanding. but here are a few things that caused some confusion. It's unclear what arguments defresolver takes. Ideally i could look at the function signature and at least have idea. I'm not sure why are we passing what seems to be a global param attached to any particular "attribute" (e.g (':b {:foo "bar"}) , please correct if me if this is the wrong term). In this example it's taking the first instance of the param and ignoring the second. (p.eql/process (pci/register param-item) {:id 1} ['(:b {:foo "car"}) '(:a {:foo "bar"}) ]);; => {:b {:foo "car"}, :a {:foo "car"}} Why not have it as a completely separate argument? I suspect there is a good reason it just hasn't clicked for me yet. Again, my goal was just to learn. My primary confusion was because as the oringal example shows, i was getting nil as where the same code now evals to {:a/b 1} the expected output. Ty for taking the time to answer my questions.#2021-08-2120:17wilkerlucioparams are per resolver, it makes more sense if understand how the planning works (https://pathom3.wsscode.com/docs/planner), it goes scanning the query figuring out what resolvers get called, when it sees an attribute it adds the params to that resolver call (that's why it comes from the env, the pco/params helper will find the node representing that resolver execution and pull the params definition from there)
#2021-08-2120:18wilkerluciothat said, params are mostly used in resolvers with a single list in the output, to do things like pagination or some basic filtering, if the param is intended for something else, you should consider making it an input instead, inputs are a preferable way in general to give more data then params is (since they can participate in the resolution process, meaning their implementation remains open and can evolve)#2021-08-2201:21nivekuilcleaned up the pathom-based dependency injection framework I had talked about in this channel earlier. no release yet but please give the SHA a try https://github.com/nivekuil/nexus#2021-08-2209:56pithylessInteresting use-case for pathom! One thing I'm not clear on after reading the README: can I run two instances of :my.ns/server with different ports? My first impression is that there is a missing layer of indirection, because the fully-qualified names represent both the identity of a specific instance as well as the reference to the implementation.#2021-08-2210:46nivekuilthanks for the feedback! the keys are for maps that are stored in the pathom env, so I think that would be possible, you'd just have to call nx/init twice and keep both references to the returned env around. I think it'd be pretty much the same as how you'd do it in integrant? not sure what the use case would be though#2021-08-2213:23pithylessIt's good to know you can just call nx/init twice and have multiple envs (so no issue with global state), but these 2 envs would be share-nothing envs (i.e. you couldn't have a component dependent on two such servers.. since they're in separate envs and know nothing about each other). Integrant tries to resolve this issue with composite keys and composite references (https://github.com/weavejester/integrant#composite-keys).#2021-08-2213:26pithylessWe've had use-cases where we want to spin up several instances of some component and give it unique global names; but then have an explicit separate step to bind those to local names for dependent components. (i.e. we can spin up both :some.ns/handler-1 and :some.ns/handler-2 and then explicitly pass in a dependency graph that says :component/x has a dependency: e.g. {:handler :some.ns/handler-1} )#2021-08-2213:27pithylessThis is why we ended up not using integrant, but a homegrown solution that borrows a lot from https://github.com/juxt/clip and https://github.com/walmartlabs/schematic.#2021-08-2213:29pithylessThere's no right way to skin the DI-cat, so I'm always interested in learning about how others approach this problem. :)#2021-08-2213:37nivekuilthat's an interesting problem. how many instances are we talking about, are they manually differentiated, or programatically generated?#2021-08-2213:39nivekuilwhy not just use clip?#2021-08-2213:49nivekuilafaict the best you can do with nexus now is just to make a different component with different inputs that shares implementation e.g. a common function for the body (so sharing implementation would be done at a level below the component, unlike in integrant where the component itself is the implementation to be shared). This seems mostly ok if it's only a handful of components, and in theory you could dynamically generate resolvers with a macro but that could be dicey. I haven't looked into the new dynamic resolvers much, maybe there's something there, especially wrt. passing in new dependencies.. intuitively it seems like pathom would be pretty good at dealing with fine-grained data sharing/differentiation requirements#2021-08-2214:07nivekuilI think the easiest change to facilitate that use would be to allow overriding pco/output , and then you could pass in a vector of inputs and return a pco/output with multiple keys#2021-08-2214:19nivekuilactually, aren't derived keywords essentially a join? I've only tested with flat queries but I wonder if what you describe is already addressed thanks to pathom's existing behavior#2021-08-2306:13lsenjovI'm a bit stuck on a use case. I'm wanting to take the example from https://blog.wsscode.com/pathom/v2/pathom/2.2.0/connect/resolvers.html#_n1_queries_and_batch_resolvers , but instead of having the total stored under :items, instead bring it up to the top level map next to :items as something like :total-items. I'm not sure how to add the nested query into the inputs set on a resolver, or even if it's possible I've been playing with
(pc/defresolver total-items-resolver
  {::pc/input #{[{:items [:number]}]}
   ::pc/output [:total-items}
  ...)
and request with
(parser {} [:total-items])
which in my mind matches the list-things resolver from the example, then can operate on that. So far I can't figure out how to do it. Is there a way to do this currently or am I barking up the wrong tree?
#2021-08-2313:36wilkerluciohello, it looks like what you want is nested inputs, Pathom 2 doesn't have support for that#2021-08-2313:37wilkerlucioPathom 3 does support it: https://pathom3.wsscode.com/docs/resolvers#nested-inputs#2021-08-2405:24lsenjovOh that works perfectly! Thank you so much!#2021-08-2323:49stuartrexkingAre pathom 2.4 resolvers and mutations directly invokable like they are in pathom 3?#2021-08-2323:49stuartrexkingI want to unit test them in isolation (without a parser)#2021-08-2323:56stuartrexkingThe answer is no, they are just maps.#2021-08-2423:26wilkerlucio@stuartrexking not directly callable, but from resolvers you can get the fn from the ::pc/resolve key, and ::pc/mutation for mutations#2021-08-2423:27stuartrexking@wilkerlucio Thanks. That’s what I’m doing.#2021-08-2523:28markaddlemanI'm wondering the best way to model this use case: The UI needs to display a bunch of tables. The particular columns of each of the table are controlled by the server. The UI receives a data structure that fully describes the table. Something like
{:table/cols [{:table.col/id 1, :table.col/label "Column 1", :table.col/attribute :attr-1},
              {:table.col/id 2, :table.col/label "Column 2", :table.col/attribute :attr-2}]
 :table/rows [{:table.row/id 1, :attr-1 "in col 1", :attr-2 "in col 2"}]}
I figure the UI will query for the table with EQL [{:some/identifier [:table/cols, :table/rows]}] I'm stumbling over how to model the resolvers in the server. The :some/identifier is straightforward:
(pco/defresolver data []
  {::pco/output [:some/identifier]}
  {:some/identifier [{:some.identifier/id     1
                      :some.identifier/attr-1 "in col 1"
                      :some.identifier/attr-2 "in col 2"}]})
The resolver for [:table/rows :table/cols] is not. I'd like to have a single resolver that can take arbitrary input and reformat the data into rows and cols as the client is expecting. I'm getting hung up on the "resolver that can take arbitrary input" - This seems exactly counter to Pathom's philosophy. I thought I could use a wildcard for a resolver input but that does not work. Any thoughts?
#2021-08-2523:43markaddlemanMaybe this is better handled as a plugin than as a resolver?#2021-08-2609:06cyppanI think it would be nice to have an EQL syntax check (in Pathom 3) in p.eql/process and p.a.eql/process, I’ve lost 10mn with a meaningless stacktrace clojure.lang.Keyword cannot be cast to java.util.Map$Entry at java.lang.Thread.run(Thread.java:748) I had a '(:k (pathom/as :renamed)) instead of '(:k {pathom/as :renamed}) .. :face_palm:#2021-08-2609:36jmayaalv@cyppan did you try checking it with the eql spec https://github.com/edn-query-language/eql#clojure-specs was the output any good? i agree a syntax check plugin could come handy though šŸ™‚#2021-08-2609:53cyppanDidn’t know about that, it will do the job indeed thank you#2021-08-3000:46caleb.macdonaldblackRegarding authorization & permissions, almost every resolver that takes input needs to do an permissions check to see if the requesting client/user is permitted to retrieve the resource. Does this makes sense or am I doing something wrong?#2021-08-3000:48caleb.macdonaldblackPerhaps I could whitelist my pathom inputs coming in from the client?#2021-08-3003:21wilkerlucioa good place to isolate this kind of logic is to use a transform function (https://pathom3.wsscode.com/docs/resolvers/#resolver-transform), this looks like this:
(ns com.wsscode.scrapper.auth-demo
  (:require [com.wsscode.pathom3.connect.operation :as pco]
            [com.wsscode.pathom3.connect.indexes :as pci]
            [com.wsscode.pathom3.interface.eql :as p.eql]))

(defn auth-transform [resolver-config]
  (update resolver-config
    ::pco/resolve
    (fn [resolve]
      (fn [{::keys [authorized?] :as env} input]
        (if-not authorized?
          (throw (ex-info "Unauthorized" {})))
       
        (resolve env input)))))

(pco/defresolver user-by-id []
  {::pco/transform auth-transform}
  {:user/name "foo"})

(def env
  (pci/register user-by-id))

(comment
  ; fails
  (p.eql/process-one
    (pci/register user-by-id)
    :user/name)

  ; succeed
  (p.eql/process-one
    (pci/register {::authorized? true} user-by-id)
    :user/name))
#2021-08-3003:22wilkerlucioanother approach is to write a plugin to wrap all resolver calls, this way you can affect the whole system at once, but I find this approach a bit overkill, and adds overhead to resolvers that could avoid it.#2021-08-3003:23wilkerlucioanother way is to categorize any resolver that is fetching restricted data to be a special kind of resolver, in the sense that you made a custom constructor (you can wrap the macro defresolver with your own macro), so you use that macro, and at that level you ensure its setup with the security checks in place#2021-08-3003:28wilkerlucioI believe when you said "take input" you mean identifier inputs, usually :user/id, :group/id, etc... IME these are usually a small part compared to the number of other attributes that might be inputs for transformation, so a nice way to write some generic code for it is to use a consistent heuristic (like: every attribute :*/id is an indentifier and requires checks), or keeping a set with the list of which they are, so you can process then generically#2021-08-3023:56caleb.macdonaldblack@U066U8JQJ Your last comment describes my issue. I thought about processing the client inputs server-side stripping out data that isn’t whitelisted, and then also checking the user has permission to access resources the data identifies. I could get pretty far and save time doing this, however there are use-cases in my permissions model that require me to go further. For example just because one user type can access one resource, doesn’t mean they can access another resources referenced by the first. initial input sanitation won’t cover this scenario and I would need to go further. I’m playing around with the idea that resolvers in my client accessible service-side index cannot trust its inputs and must use ā€œUserā€ data in the env map to return results the user is authorised to retrieve. The caveat being my data access layer needs to use the user object to determine what it can return. Mostly this just looks like table joins to the permissions tables. So far that is working for me but it’s more work than a more generic solution.#2021-08-3004:38stuartrexkingWhy would params not show up in (-> env :ast)?#2021-08-3019:58wilkerluciodepends at which point the param is, and where you are looking for it (I mean, at which resolver, and how its that related to the param), can you send an reproducible example?#2021-08-3121:15JAtkinsP2 or P3? I think it’s been removed in P3, @U066U8JQJ?#2021-08-3121:19wilkerluciogood point, in P3 you should use the (pco/params env) instead of (-> env :ast)#2021-08-3121:20wilkerluciobut I guess the issue may be the more common misunderstanding about params location#2021-09-0206:41JAtkinsWhere do params appear? I just ran into this misunderstanding myself with p3. When attaching a param to a property, the params are not passed too resolvers required to fulfill that first requirement. e.g.
(defresolver a [env _] (pco/params env) #_=>nil {:a 1})
(defresolver b [env {:keys [a]}] (pco/params env) #_=>{:param :param} {:b (inc a)})

(p.eql/process env ['(:b {:param :param})])
This is on the FAQ page of p3 docs, but currently blank.
#2021-09-0211:45wilkerlucioparams do not flow to dependencies, the behavior you sent is the expected one#2021-09-0212:17wilkerluciothat FAQ page isn't released, haha, weird it got indexed on search#2021-08-3004:39stuartrexkingI have a query with params but they don’t appear in the env :ast#2021-08-3004:48stuartrexkingLooks like this is my issue https://github.com/wilkerlucio/pathom/issues/93#2021-09-0101:32caleb.macdonaldblack@wilkerlucio What inspired you to create Pathom?#2021-09-0102:40souenzzoI'm also curious to read Wilkers response but the part that I know: Pathom started from om.next ideas (server part) https://github.com/omcljs/om/wiki/Quick-Start-(om.next)#parsing--query-expressions I also recommend to watch one of David Nolen videos on youtube about om.next ideas. om.next was never finished. Then emerged #fulcro as the spiritual successor for the frontend, and #pathom to the backend "Pathom connect" is a original idea from Wilker as far I know.#2021-09-0102:58caleb.macdonaldblack@U2J4FRT2T Same. I’m really excited about Pathom. It seems like such a simple idea of which surely has somewhat been inspired by other ideas. I’m curious about what is original and what isn’t. I’m vaguely understanding that is has some inspiration from Logic Programming as mentioned on the Pathom website but it seems to me it takes it a step further than that. Either that or I need to spend some more time on logic programming#2021-09-0113:58souenzzoAfter ~3 years working with pathom, programming without it feels like "programming without a GC" We need to take care about if a value is fetched before pass it to another function With pathom, we just require what we want and use it. We don't need to care about the "fetch lifecycle"#2021-09-0118:56wilkerluciohello @U3XCG2GBZ, as @U2J4FRT2T said, Pathom started as some extensions I made to create parsers for Om.next (so the name: Path Om) early inspirations to go on the idea are @U050B88UR presentations like: https://www.youtube.com/watch?v=fYLM5nCIBQg in the beginning there connect idea didn't exist, I was just trying to make easier to write simple parsers, the main idea was to have a single implementation for every attribute in the system. When I had the connect idea, I was implementing fetching for basic customer data, back then it looked something like this:
(defn read-customer-data [env]
  (let [{:keys [customer/id]} (p/entity env)
        response (cached-read env
                   (str "" id))]
    (get 
      (adapt-customer response) 
      (-> env :ast :key))))

(def customer-readers
  {:customer/name
   read-customer-data
   
   :customer/email
   read-customer-data
   
   :customer/account-id
   read-customer-data
   
   ...})
By the time I wasn't very happy about that, for each request we need to give the full implementation, but the turning point was that on top of getting user by ID, I also needed to pull him by SSN, which made the request fn look like this:
(defn read-customer-data [env]
  (let [{:keys [customer/id customer/ssn]} (p/entity env)
        response (cached-read env
                   (if ssn
                     (str "" id)
                     (str "" id)))]
    (get
      (adapt-customer response) 
      (-> env :ast :key))))
The change might not seem so bad, until you realise the endpoint for SSN doesn't have all the same fields of the endpoint by ID, so now we need 3 fetch implementations, one that supports both ID and SSN, one just for ID, one just for SSN... By that time we were experimenting if the idea would fit, and IMO this wouldn't scale well for the whole organization, I felt it needed to be better, easier to handle Looking at that I though: what if I index the attributes with some fns, and have some code to traverse this index? And that was the beginning of the Connect idea, wasn't something I saw anywhere else, but an emergent solution for the problem I had in front of me. The first versions were very crude (had lots of issues with circle references for example), you can still it, its the reader on Pathom 2 (https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/connect.cljc#L709). These connect readers had a lineage: reader - the first experimental one, very eager, it wasn't used for long since it has a bunch of pitfals (inneficient, cycle errors...) reader2 - this was the first time Pathom did some kind of planning, for a single attribute it traverses the indexes to find a sequence of resolvers to call (or to find the path isn't possible) reader3 - this has a more advanced planning, that generates an execution graph, Pathom 3 uses an evolved version of this algorithm And the Pathom lineage: Pathom 1 - this version depended on om.next, used the basic EQL parser and parser construct directly from the om.next code Pathom 2 - removed dependency with Om.next, EQL was extracted in its own library (so both Pathom and Fulcro could share it), and the parser infra-structure was copied into Pathom Pathom 3 - a complete redesign, abandoning the parser concept from Om.next and making its own processing method, that aligns with the knowledge from Pathom 2 and make connect the center of it (instead of an extension, like it is on Pathom 2) So the connect idea is mostly original, I only learned about RDF much later and was surprise how the modeling was similar, but not by accident, my modeling was inspired by Spec/Datomic, which are inspired in RDF to begin with.
#2021-09-0306:55caleb.macdonaldblackHi @wilkerlucio. Thanks for such a detailed response. I really appreciate it. The David Nolen talk is great and describes similar issues to what I’m experiencing in my software.#2021-09-0106:10JAtkinsI think I found a bug with p3 nested input planning:
(defresolver nested-output-1 
  [_ _]
  {::pco/output [{:nested-output-1 [:basis-key]}]}
  {:nested-output-1 {:basis-key 10}})

(defresolver derived-key 
  [{:keys [basis-key]}]
  {:derived-key (inc basis-key)})

(defresolver nested-output-2 
  [_ {{:keys [basis-key derived-key] :as nested-output-1} :nested-output-1}]
  {::pco/input [{:nested-output-1 [:basis-key
                                   :derived-key]}]
   ::pco/output [{:nested-output-2 [:basis-key
                                    :derived-key
                                    :output-2-key]}]}
  {:nested-output-1 (assoc nested-output-1
                           :output-2-key (+ basis-key derived-key))})

(let [env ((requiring-resolve 'com.wsscode.pathom.viz.ws-connector.pathom3/connect-env)
           (pci/register [nested-output-1 nested-output-2 derived-key])
           {:com.wsscode.pathom.viz.ws-connector.core/parser-id "TRACER"}) ]
  (p.eql/process
   env 
   [{:nested-output-2 [:basis-key
                       :derived-key
                       :output-2-key]}]))

;; =>

;; Execution error (ExceptionInfo) at com.wsscode.pathom3.connect.runner/check-entity-requires! (runner.cljc:733).
;; Required attributes missing: [:nested-output-2] at path []
Run on latest 2021.08.14-alpha
#2021-09-0108:37donavanYour stated output in nested-output-2 doesn’t match the actual output; :nested-output-2 in ::pco/output but the map key is :nested-output-1#2021-09-0108:38donavanThat does seem to be the issue from a glance… There are currently limitations with planning and nested output currently in P3, I’m going to take a stab at implementing Wilker’s suggested solution today but it doesn’t look like it’s related to this.#2021-09-0111:23JAtkinsHmm… I think I was too tired when I wrote that. Good catch :thumbsup:#2021-09-0113:15sheluchinI'd like to contribute a small change to the docs:
diff --git a/docs/index.adoc b/docs/index.adoc
index 9bf035a6..863dd36b 100644
--- a/docs/index.adoc
+++ b/docs/index.adoc
@@ -1969,6 +1969,11 @@ your index out.
 
 Using this you can control what gets out to the explorer.
 
+Note that you may need to take additional steps to make sure the index doesn't trip
+when coming across functions or other values that cannot be serialized for network
+transfer without special handling. See the <<Fixing transit encoding issues,Transit issues>>
+section for a snippet that implements the filtering on the resolver.
+
 === Visualizing your index
 
 Here you will find some ways to visualize your index.
@@ -2033,6 +2038,20 @@ include other things that are not possible to encode with transit by default. We
 you setup a default write handler on Transit so it doesn't break when it encounter a value
 that it doesn't know how to encode.
 
+Another approach is to filter out the `:resolve` and `:mutate` functions when setting up
+the index explorer resolver:
+
+[source,clojure]
+----
+(pc/defresolver index-explorer [{::pc/keys [indexes]} _]
+  {::pc/input  #{:com.wsscode.pathom.viz.index-explorer/id}
+   ::pc/output [:com.wsscode.pathom.viz.index-explorer/index]}
+  {:com.wsscode.pathom.viz.index-explorer/index
+   (p/transduce-maps
+     (remove (comp #{::pc/resolve ::pc/mutate} key))
+     indexes)})
+----
+
 If you are running Pathom in Clojure, then you also need to know there is a bug in the
 current Clojure writer, it doesn't support default handlers (although the docs say it does).
Should this be a PR to master? Also, when I run antora docs-dev.yml, the change set is quite large:
$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   docs/index.adoc
        modified:   docs/v2/404.html
        modified:   docs/v2/index.html
        modified:   docs/v2/pathom/2.2.0/cljs-specs.html
        modified:   docs/v2/pathom/2.2.0/connect.html
        modified:   docs/v2/pathom/2.2.0/connect/basics.html
        modified:   docs/v2/pathom/2.2.0/connect/connect-mutations.html
        modified:   docs/v2/pathom/2.2.0/connect/exploration.html
        modified:   docs/v2/pathom/2.2.0/connect/indexes.html
        modified:   docs/v2/pathom/2.2.0/connect/readers.html
        modified:   docs/v2/pathom/2.2.0/connect/resolvers.html
        modified:   docs/v2/pathom/2.2.0/connect/shared-resolvers.html
        modified:   docs/v2/pathom/2.2.0/connect/thread-pool.html
        modified:   docs/v2/pathom/2.2.0/core.html
        modified:   docs/v2/pathom/2.2.0/core/async.html
        modified:   docs/v2/pathom/2.2.0/core/entities.html
        modified:   docs/v2/pathom/2.2.0/core/error-handling.html
        modified:   docs/v2/pathom/2.2.0/core/errors.html
        modified:   docs/v2/pathom/2.2.0/core/getting-started.html
        modified:   docs/v2/pathom/2.2.0/core/mutations.html
        modified:   docs/v2/pathom/2.2.0/core/parsers.html
        modified:   docs/v2/pathom/2.2.0/core/path-track.html
        modified:   docs/v2/pathom/2.2.0/core/placeholders.html
        modified:   docs/v2/pathom/2.2.0/core/readers.html
        modified:   docs/v2/pathom/2.2.0/core/request-cache.html
        modified:   docs/v2/pathom/2.2.0/core/trace.html
        modified:   docs/v2/pathom/2.2.0/graphql.html
        modified:   docs/v2/pathom/2.2.0/graphql/edn-to-gql.html
        modified:   docs/v2/pathom/2.2.0/graphql/fulcro.html
        modified:   docs/v2/pathom/2.2.0/introduction.html
        modified:   docs/v2/pathom/2.2.0/modeling.html
        modified:   docs/v2/pathom/2.2.0/other-helpers.html
        modified:   docs/v2/pathom/2.2.0/plugins.html
        modified:   docs/v2/pathom/2.2.0/rationale.html
        modified:   docs/v2/pathom/2.2.0/sugar.html
        modified:   docs/v2/pathom/2.2.0/upgrade-guide.html
        modified:   docs/v2/sitemap.xml
Should I only commit the changes to docs/index.adoc in this case?
#2021-09-0113:20wilkerlucioyes, please make a pr#2021-09-0113:26sheluchinThanks! https://github.com/wilkerlucio/pathom/pull/200#2021-09-0313:30wilkerluciohello, glad to announce a few new documentation pages on Pathom 3! • https://pathom3.wsscode.com/docs/foreign • https://pathom3.wsscode.com/docs/dynamic-resolvers • https://pathom3.wsscode.com/docs/integrations/graphql • https://pathom3.wsscode.com/docs/tutorials/graphql-integration This is the beginnings of the https://www.youtube.com/watch?v=IS3i3DTUnAI vision landing on Pathom 3!#2021-09-0314:09markbastianMan, foreign envs/resolvers is about the coolest thing ever.#2021-09-0314:59wilkerlucio#2021-09-0316:42sheluchinI'm confused why a note-cards-resolver is being called when it doesn't promise the desired fields in ::pc/output:
(api-parser
    [{[:note/id 1]
      [{:note/card-list [:card-list/id]}]}])
(pc/defresolver note-id->card-list-id-resolver
  [{:keys [crux] :as env}
   {:note/keys [id]}]
  {::pc/input #{:note/id}
   ::pc/output [{:note/card-list [:card-list/id]}]}
  (log/debug "note-id->card-list-id-resolver" {:note/id id})
  {:note/card-list
   {:card-list/id (q/note-id->card-list-id crux {:note/id id})}})

(pc/defresolver note-cards-resolver
  [{:keys [crux] :as env}
   {:note/keys [id]}]
  {::pc/input #{:note/id}
   ::pc/output [{:note/card-list [{:card-list/cards [:card/id]}]}]}
  (log/debug "note-cards-resolver" {:note/id id})
  #p {:note/card-list
      {:card-list/cards (get-card-ids-from-note crux {:note/id id})}})
2021-09-03T16:31:15.591Z xps INFO [sheluchin.parser:37] - Process [{[:note/id 1] [#:note{:card-list [:card-list/id]}]}]
2021-09-03T16:31:15.593Z xps DEBUG [sheluchin.resolvers:88] - note-id->card-list-id-resolver #:note{:id 1}

2021-09-03T16:34:50.814Z xps INFO [sheluchin.parser:37] - Process [{[:note/id 1] [#:note{:card-list [:card-list/id]}]}]
2021-09-03T16:34:50.816Z xps DEBUG [sheluchin.resolvers:97] - note-cards-resolver#:note{:id 1}
#p[sheluchin.resolvers/note-cards-resolver:92]
#:note {:card-list #:card-list {:cards (get-card-ids-from-note crux #:note {:id id})}}
=> #:note {:card-list #:card-list {:cards [#:card {:id 1}]}}
Can anyone help me understand what's happening here? It looks to me like I'm ultimately requesting :card-list/id that's related to a :note/id. A note looks like this:
{:note/id 1 
 :note/card-list {:card-list/id 2}}
#2021-09-0321:41wilkerlucioPathom doesn't take the nested output in consideration, for pathom you just have 2 options for :note/card-list#2021-09-0321:42wilkerlucio@U0VP19K6K is doing some work to make this be a consideration in Pathom 3#2021-09-0321:42wilkerluciobut Pathom 2 awareness of nested things like this is more limited#2021-09-0713:51sheluchin@U066U8JQJ thank you#2021-09-0813:22sheluchin@U066U8JQJ what is the recommended approach for dealing with this limitation? Should resolver output descriptions generally be flat/without nesting? Hmm... I see the docs explain that nested descriptions are still useful for autocomplete and automatic testing. I guess that means that nested output descriptions should be used, but that if the nested data needs to be exposed to the graph, those keys also need to be included as top-level keys in their own resolvers?#2021-09-0319:14azSilly question, is there a use case for using Pathom to query Datascript on the front-end for certain situations? Resolving on Datascript indexes directly? Reason I ask is I’m hitting a wall with some recursive queries in Datascript. I’m sure it’s something in the way I’m writing the query, but it got me wondering if Pathom would fit in this situation if there are deeply nested structures. Is there a big difference between EQL pull syntax in Pathom and Datascript?#2021-09-0319:31lilactownthe pull syntax for DataScript is different than Pathom#2021-09-0319:45azWould pathom make sense in this instance or am I misunderstanding Pathom?#2021-09-0319:52lilactownpathom probably won't help you write faster datascript queries#2021-09-0321:44wilkerlucioPathom isn't going to be faster, but I have cases that I used resolvers to query on datascript, the advantage is that you can extend the system with things that are not in datascript, its a bit more advanced, but using dynamic resolvers you can make pathom aware of what to delegate down to datascript#2021-09-0321:44wilkerluciothe closest example to that is the datomic integration for Pathom 3#2021-09-0321:44wilkerluciohttps://github.com/wilkerlucio/pathom3-datomic#2021-09-0322:20azThank you @U066U8JQJ sounds great.#2021-09-0319:33lilactownI think a library that implemented EQL on top of datascript would be p cool#2021-09-0321:37henrikhttps://github.com/lilactown/autonormal implements EQL-style pull, and harmonizes quite well with Pathom.#2021-09-0322:30azOh this looks fantastic. Thank you @U06B8J0AJ @U4YGF4NGM#2021-09-0416:59lilactownyeah, it's not going to interop with datascript, but if you're willing to replace datascript - it's a pretty good solution šŸ™‚#2021-09-0417:00lilactowni'm secretly working on a datalog impl for it, but it probably won't ever have complete parity with datascript's features#2021-09-0602:06azThat’s exciting#2021-09-0417:05mauricio.szaboHi, @wilkerlucio. I was reading about the dynamic resolvers but I couldn't really understand what it does... can you give an example, maybe something that don't depend on external systems so it's easier to get?#2021-09-0417:11wilkerluciothe thing is that dynamic resolvers are about external systems, its a way to ā€œmaskā€ part of a subquery to send to some external system (or to process the eql in some custom way, like walkable does)#2021-09-0417:11wilkerluciobut I’ll see if I can coin some other examples#2021-09-0418:53eoliphantHi, I’m having a weird problem with Viz (im on the latest 2021.7.16 and 2021.07.15-1 of the connector). Traces for whatever reason just don’t show up, even though everything else seems fine. If i run something like
(-> (p3/parser  {}
        [`(:location/suggestions {:search "fortleza"})])
    meta
    ::pcr/run-stats)
I get the run-stats in the repl, the request and response show up in Viz, but never any trace (or graph and node detail) info.
#2021-09-0419:46wilkerlucioare you using p3 namespace connector?#2021-09-0419:50eoliphanthmm let me confirm#2021-09-0419:52eoliphantlooks like it
(:require [com.wsscode.pathom.viz.ws-connector.core :as pvc]
            [com.wsscode.pathom.viz.ws-connector.pathom3 :as p.connector] 

...

       (log/error error :node node)))})
    (p.connector/connect-env {::pvc/parser-id `env}))
#2021-09-0419:54wilkerlucionot sure them, can you make a reproducible example?#2021-09-0419:58eoliphantit’s in a big project now lol, but yeah let me see if i can create something stripped down, will shoot it over if it persists#2021-09-0420:21eoliphantok.. so i suspect there’s some dependency weirdness. I created a minimal project, and just copied in the code from the example in the docs and of course it worked fine. then copied the same code into my code base, and same issue. so gonna compare the resolved deps#2021-09-0516:05eoliphantok so. i’m an idiot lol. after, getting my test project to exact dependency parity, etc, traces were working fine. So then i stripped down my user.clj in my actual project to require nothing, and the traces magically showed up. long story short, lol, because of an issue with the transit version on older versions of datomic cloud, i’d put in a hack to define cognitect.transit/write-meta as null to keep things from blowing up, along with a note to take it out as soon as the server dependency issue got fixed. Sorta didn’t get around to the last part lol#2021-09-0708:59henrikI'm getting a weird problem in Pathom 3. On running (pco/params env), it gives me {:com.wsscode.pathom3.connect.operation/optional? true}, which are not my parameters. If I run (get-in env [::pcp/graph ::pcp/source-ast :params]) I can find my actual parameters. Has anyone seen this?#2021-09-0809:31cyppanhow do you specify your params? In a placeholder? I don’t know if it’s your problem but if you specify the data in a placeholder (https://pathom3.wsscode.com/docs/placeholders) they will be available as resolver inputs but not as params (https://pathom3.wsscode.com/docs/resolvers#parameters) Regarding the :com.wsscode.pathom3.connect.operation/optional?, it’s a param injected when you use (pco/? on your eql keys:
(pco/? :key)
=> (:key #:com.wsscode.pathom3.connect.operation{:optional? true})
#2021-09-0817:22wilkerlucioharder to say without the full example#2021-09-0807:29HukkaCouldn't find any talk about this in google; is there some way to implement api quotas in Pathom? With restish api it's easy enough to count requests, but with graphs a single request can be small or can get pretty much everything at the same time.#2021-09-0809:42Björn EbbinghausWell. That's highly dependent on your needs. And not directly a concern of pathom. You could count request, calls of mutations, access to attributes... Whatever. Maybe you could look into how existing, big GraphQL APIs handle rate limiting? Like from GitHub? https://docs.github.com/en/graphql/overview/resource-limitations#rate-limit#2021-09-0810:18HukkaI'm guessing that it would need some kind of context in Pathom, where different resolvers can accumulate the cost, and abort or truncate the request if it gets too costly. Of course if the cost could be calculated based on the query without knowing the data (so depth, without knowing the width of the data), a proxy could already handle it in front of actual pathom#2021-09-0817:34wilkerlucioI think making it by resolver is a reasonable path, since you can kind estimate the cost of each#2021-09-0908:10HukkaIn the github example, they know the cost of query since all the levels are limited in width. But I was wondering if it's possible to keep track of the result width while resolving, and somehow abort. Recording the cost from realized result and stopping future queries if the quota ran out would be easy enough, I guess#2021-09-1014:15Björn EbbinghausYou can write a plugin for this. Not the same, but maybe an example you could work of: I wrote an access plugin that keeps a set of "allowed" inputs for a request. For each resolver call, I check if the input of the resolver is in my allowed set. If it isn't, I check if it should be allowed and if so, add it to the set. If not, I don't run the resolver. https://github.com/hhucn/decide3/blob/master/src/main/decide/server_components/access_plugin.cljc Why could this be relevant for your situation? 1. It has a per-request store. (In your case, you could record cost.) 2. It uses the store to decide whether to run a resolver or not. (You could check: Am I over the quota?) #2021-09-1019:18wilkerluciohello @tomi.hukkalainen_slac, here is an example on how to make a plugin to handle resolver quotas in Pathom 3: https://gist.github.com/wilkerlucio/d9cdffb9e30570846a4297057e2dba6f#2021-09-1019:20wilkerlucioif you are using Pathom 2, things are very similar, just a different plugin entry point#2021-09-1111:24HukkaThat's great! I'll read more about plugins when I'm a bit more rested, Monday latest#2021-09-1019:18wilkerluciohello @tomi.hukkalainen_slac, here is an example on how to make a plugin to handle resolver quotas in Pathom 3: https://gist.github.com/wilkerlucio/d9cdffb9e30570846a4297057e2dba6f#2021-09-1010:49Jakub Holý (HolyJak)I stumbled upon a mystery in Pathom 2, using Fulcro Inspect EQL tab. I have a mutation that throws an exception and thus this works as expected:
[{(com.example.mutations/create-random-thing {:tmpid 123}) [:com.wsscode.pathom.core/errors]}]
; =>
com.example.mutations/create-random-thing
 {:com.wsscode.pathom.core/reader-error "fake error - {}"}}
but if I add :p/errors to the transaction (which is actually likely unnecessary / useless, as well as the ::p/errors inside the mutation join), suddenly the output changes in a weird way:
[:com.wsscode.pathom.core/errors
 {(com.example.mutations/create-random-thing {:tmpid 123}) [:com.wsscode.pathom.core/errors]}]
; =>
{com.example.mutations/create-random-thing
 {:com.wsscode.pathom.core/reader-error
  "Mutation not found - {:mutation com.example.mutations/create-random-thing}"},
 :com.wsscode.pathom.core/errors :com.wsscode.pathom.core/not-found}
Why do I suddenly get "Mutation not found" error?! (FYI this is related to https://github.com/fulcrologic/fulcro/pull/486) šŸ™
#2021-09-1019:05wilkerluciothis is a bug in Pathom Viz in terms of handling mutations, its old but its been a while since someone stumble on it, I made an issue to track it: https://github.com/wilkerlucio/pathom-viz/issues/69#2021-09-1019:18wilkerlucio#2021-09-1120:46Jakub Holý (HolyJak)@wilkerluciošŸ™ My assumption was that if I send a mutation with a query to Pathom, it will filter out from the mutation's output everything that has not been asked for. However this seems to not be the case and Pathom does return also other stuff returned by the mutation, namely :tempids . This is actually what I desire, so it is just fine, I only want to confirm that this is indeed expected behavior. Pathom 2.3.1. For details, see https://github.com/fulcrologic/fulcro/issues/484#issuecomment-917477147, if necessary. Thank you!#2021-09-1308:07Jakub Holý (HolyJak)It worked b/c my Pathom config had {::p/env {::pc/mutation-join-globals [:tempids], ...}}#2021-09-1313:51wilkerlucioAwesome! As a rule of thumb, if you get some result running a query in the REPL, you should be able to get the same result in Pathom Viz#2021-09-1313:51wilkerluciowhat I remember is that when using mutation joins, Pathom Viz may not follow it up correctly, I need to investigate that closer#2021-09-1314:11plinsis there a way of having nested batching resolvers? https://github.com/wilkerlucio/pathom/issues/121 suggests thats not the case šŸ˜• I wonder if anything changed with pathom3, or if there is a tool like https://github.com/graphql/dataloader/issues/79#issuecomment-361758674/https://www.juxt.pro/blog/superlifter#2021-09-1314:23wilkerluciohello, this is a non-issue in Pathom 3, the batch mechanism there uses a different idea and can handle cases like nested and even entities that are disperse across the request#2021-09-1314:27plinsexcellent news šŸ™‚#2021-09-1314:40Ryan JerueOn a related note, is there any sort of upgrade doc/guide for pathom2 šŸ‘‰ pathom3?#2021-09-1314:45wilkerlucionot yet, but I would love to hear feedback and assist anyone wanting to do it#2021-09-1314:45wilkerlucioresolvers / mutations are mostly portable, but there may be some work to port plugins#2021-09-1402:39caleb.macdonaldblackIs there a tool I can use to visualise my pathom3 index?#2021-09-1402:51caleb.macdonaldblackFound it: https://pathom3.wsscode.com/docs/debugging/#debug-with-pathom-viz#2021-09-1403:04caleb.macdonaldblackIn the pathom viz tool I get: > Entity data requires Pathom 2.4.0+ > Or latest Pathom 3 from Git I’m using [com.wsscode/pathom3 ā€œ2021.08.14-alphaā€] do I need to be newer than that?#2021-09-1423:18wilkerluciostill having issues? a common one is to use the wrong namespace to start the connector#2021-09-1412:27souenzzo@caleb.macdonaldblack no. when this was written, the only way to get pathom3 was from git. these alpha releases are pretty new.#2021-09-1421:10plinsIm trying to follow some pathom3 examples
(def users-db
  {1 #:acme.user{:name     "Usuario 1"
                 :email    "
so my code looks like
(def products {1 {:product/name           "Moog Voyager XL"
                  :finance/price-in-cents 400000}
               2 {:product/name           "Roland V-Drum"
                  :finance/price-in-cents 40000}
               3 {:product/name           "RME UFX II"
                  :finance/price-in-cents 100000}
               4 {:product/name           "UAD Satellite Octo"
                  :finance/price-in-cents 100000}
               5 {:product/name           "Beer"
                  :finance/price-in-cents 300}})

(pco/defresolver product-id->product
  [_ctx {:product/keys [id]}]
  [{::pco/input  [:product/id]
    ::pco/output [:product/name
                  :finance/price-in-cents]}]
  (get products id))
but Im getting
Syntax error macroexpanding com.wsscode.pathom3.connect.operation/defresolver at (src/reify/main.clj:61:1).
{:name product-id->product, :arglist [[:sym _ctx] [:map #:product{:keys [id]}]], :body [[#:com.wsscode.pathom3.connect.operation{:input [:product/id], :output [:product/name :finance/price-in-cents]}] (get products id)]} - failed: (fn must-have-output-visible-map-or-options [{:keys [body options]}] (or (map? (last body)) options)) spec: :com.wsscode.pathom3.connect.operation/defresolver-args
this solves the error
(pco/defresolver product-id->product
  [_ctx {:product/keys [id]}]
  [{::pco/input  [:product/id]
    ::pco/output [:product/name
                  :finance/price-in-cents]}]

  {:product/name (get-in products [id :product/name])
   :finance/price-in-cents (get-in products [id :finance/price-in-cents])})
but its a bit smelly, am I missing something?
#2021-09-1421:37cyppanYou should remove the vector around your {::pco/input ...} in your defresolver#2021-09-1421:38cyppan
(pco/defresolver product-id->product
  [_ctx {:product/keys [id]}]
  {::pco/input  [:product/id]
    ::pco/output [:product/name
                  :finance/price-in-cents]}
  (get products id))
#2021-09-1421:38cyppanit should solve your problem#2021-09-1514:26plinsoh! I did missed that! thanks a lot @U0CL38MU1#2021-09-1508:57Bjƶrn Ebbinghaushttps://clojure.atlassian.net/browse/CLJ-2123 in Clojure 1.11.0-alpha2 šŸŽ‰#2021-09-1511:11wilkerlucioyes!!! šŸŽ‰ #2021-09-1516:40lgessleris there an example of this in action anywhere? a little hard to understand it reading the ticket#2021-09-1516:44dehliI haven’t used it yet, but my understanding is that you can write code like below. If you use :as the code will fail b/c the namespace doesn’t exist but :as-alias will work.
(ns my-namespace.core
  (:require [my-namespace-that-doesnt-exist.core :as-alias foo]))

{::foo/bar true} ;; => {:my-namespace-that-doesnt-exist.core/bar true}
#2021-09-1616:45Drew VerleeIn Pathom viz why do some of the indexs show up as different colors in i the "Index Explorer" tab?#2021-09-1617:31wilkerluciopurple: attributes blue: resolvers orange: mutations#2021-09-1617:31wilkerluciois these colors you talking about?#2021-09-1718:55Drew Verleeyes they are ty ty#2021-09-1616:45Drew VerleeWhats an example of something that would go in Query > "Entity Data" ?#2021-09-1617:25dehliEntity data allows you to query for resolvers in pathom-viz and provide data to the resolver (or upstream resolvers). For example, let’s say you only have the following resolver:
(pc/defresolver full-name
  [{:user/keys [first-name last-name]}]
  {:user/full-name (str first-name " " last-name)})
If you were to query for [:user/full-name] as a top-level key, you wouldn’t get it back b/c :user/first-name and :user/last-name are needed as inputs (and pathom can’t get to either of those attributes to fulfill the inputs of :user/full-name ). You can inject those keys via entity data :
{:user/first-name "Jane"
 :user/last-name "Doe"}
and then when you query for [:user/full-name] you’ll get {:user/full-name "Jane Doe"}.
#2021-09-1617:32wilkerlucioyes, entity data is top level provided data#2021-09-1617:32wilkerlucionote the auto-complete (in the query part) is sensitive to it, if you change the data it will affect the auto-complete options (considering the provided data)#2021-09-1718:54Drew VerleeThanks!#2021-09-1622:19JonRI'm using Pathom + Walkable in a new SPA (not Fulcro). I'm wondering about client said query caching and have been google around the react world at stuff like https://react-query.tanstack.com/. Does anyone have any suggestions for resources or know of a cljs/pathom existing solution?#2021-09-1622:40lilactownI think Fulcro is state of the art atm#2021-09-1705:55HukkaFor simple things you could make a function that does the actual request, and memoize it#2021-09-1714:37JonR@lilactown I'm aware of but not experienced with Fulcro. I've seen a number of references to Fulcro tools (guardrails off the top of my head) in Pathom source. Any chance the client caching might be usable outside of fulcro? Otherwise I'll probably do something simple like @tomi.hukkalainen_slac suggests#2021-09-1716:01lilactownI know that there have been docs and examples of using fulcro's data management system w/o going whole hog to define components. it has looked pretty complex when I tried to understand it (having not used Fulcro in anger)#2021-09-1716:03lilactownhere's the docs on "Fulcro Raw" which seems to be that. it's very heavy on the lingo https://book.fulcrologic.com/#_fulcro_raw_version_3_5#2021-09-1716:04lilactownI would probably start with what Hukka said (that's what we're doing in our pathom + reagent/helix app). over time we hope to evolve to adopt either Fulcro raw or a home grown solution for caching#2021-09-1716:25JonRAwesome, thanks @lilactown for the info#2021-09-1716:25JonRANy chance you've tried react query with pathom etc?#2021-09-1716:26JonRWas just reading through that and thinking about trying it#2021-09-1716:28lilactownI haven't but I wouldn't discourage you from trying. you could perhaps key the query based on the top-level keyword of the EQL query you're making, and that might be a sane default#2021-09-1716:29lilactowni.e.
[{::foo/bar [:baz]}]
could become
(useQuery #js ["foo" "bar"] ,,,)
no idea if that's actually good in practice, just an idea šŸ˜„
#2021-09-1716:30lilactownactually that would probably cause issues... since any other dependent on ::foo/bar would only see :baz, even if they previously asked for more data#2021-09-1716:30lilactownanyway I'm interested to see if you come up with a good system! the benefit of react-query is it does give you control over how you cache and invalidate it#2021-09-1717:32JonRyup, having similar thoughts about coming up with an EQL to path like translation. Need to read more about how those paths are used and what they consider. Will share if I get something interesting#2021-09-1717:38wilkerlucioI’ve been playing with Fulcro Raw and I think with some extra helpers it gets to a quite simple and easy to use stack, I can share with you if you want, but its only an experimental thing (I haven’t tried to use it in a bigger way to see if there are any perf issues)#2021-09-1717:39wilkerluciothis is what a component looks like in this model (and I’m using Helix for the component building, and Fulcro Raw for state management):#2021-09-1717:39wilkerlucio
(h/defnc Main []
  (let [{:keys [app/todos] :as props}
        (frs/use-entity (frs/load {:app/id "cide"})
          {::frs/query [:app/id
                        {:app/todos
                         [:todo/id
                          :todo/title
                          :todo/done]}
                        :ui/load-error]})]
    (dom/div {:class "bp3-dark m-12"}
      (if (:ui/load-error props)
        (bp/callout {:intent "danger"
                     :title  "Error"}
          (dom/pre
            (ex-message (:ui/load-error props)))))
      (h/$ PeerConnectionDemo {})
      (dom/h1 {:class "mb-4"} "Todos")
      (h/$ NewTodo
        {:on-create #(frs/transact! props [(add-todo (assoc % :app/id "cide"))])})
      (dom/hr {:class "my-4"})
      #_ (bp/button {:onClick #(frs/load-component! props {})} "Reload")
      (for [{:todo/keys [id] :as todo} todos]
        (h/$ TodoItem {:key       id
                       :on-change #(frs/transact! props [(toggle-todo %)]
                                     {:component todo-comp})
                       :&         todo})))))
#2021-09-1717:39wilkerlucionote the frs/use-entity, that's the main thing there#2021-09-1717:40wilkerluciohere is the code for the helpers (the frs namespace): https://gist.github.com/wilkerlucio/3547da04b525f0ab6a156d472e55d69b#2021-09-1717:40wilkerlucio#2021-09-1717:40wilkerlucioapp starter:
(h/defnc AppWrapper []
  (frs/app-provider app
    (bp/hotkeys-provider
      (h/$ Main {}))))

(defn render! []
  (rdom/render (h/$ AppWrapper) (js/document.getElementById "app")))
#2021-09-1717:41wilkerlucioapp definition:
(defonce app
  (doto (rapp/fulcro-app
          {:batch-notifications (fn [render!] (rdom/unstable_batchedUpdates render!))
           :remotes             {:remote (frs/pathom-remote api/request)}})
    (frs/app-started!)))
#2021-09-1717:43wilkerluciobut starting with just plain hooks can work as well, the issue is more middle/long term, IME as it grows you may want to start sharing data across components, this is when Fulcro will add a lot of utility (also having Fulcro Inspect is great, so you can observe what your system is doing)#2021-09-1717:55JonRAwesome, thanks @wilkerlucio! I'll get more familiar with Fulcro Raw and try this out#2021-09-1718:49lilactown@wilkerlucio this is great, will read through it#2021-09-1807:03HukkaHaving used react-query quite a lot (and actually migrating the old code base to react-query) on a pure JS frontend with a "restish" (i.e. not REST at all) backend, I still would be a bit hesitant to use that with CLJS or graph based APIs, and still more hesitant when combined. I don't think it will be ergonomic, at least if you wouldn't be heavily using the more advanced stuff (time based staleness, automatic refetching, mutations with optimistic local changes). And perhaps only if you would use hooks anyway, and not introduce them to a cljs code base just to make react-query work.#2021-09-1807:04HukkaBut that is really just my first hunch. If this was relevant to me, I'd probably make some POCs, perhaps make multiple versions of one component to compare.#2021-09-1823:49JonRAppreciate the feedback @tomi.hukkalainen_slac, thanks. I'm gonna try and get a POC of what @wilkerlucio shared above working and see how that goes for now. I'm not familiar with Fulcro though so reading through some of that source and trying to grok what he shared. Definitely prefer a cljs option instead of navigating another big react dependency.#2021-09-1918:39Jakub Holý (HolyJak)I would hope that https://fulcro-community.github.io/guides/tutorial-minimalist-fulcro/index.html would be still the most effective way to learn some Fulcro, even though you would only use it for state management via the raw. namespaces.#2021-09-1918:42Jakub Holý (HolyJak)You want to understand idents, normalization, query and query composition, and load!, I guess#2021-09-2412:29souenzzoHey @wilkerlucio, I was prototyping ideas with pathom3, using a :pathom/as alias plugin, and I end up in a cache issue There is something that I can do to avoid cache in that case? I tried ::pcr/resolver-cache* nil and (reify p.cache/CacheStore (-cache-lookup-or-miss [this k f] (f))) https://gist.github.com/souenzzo/ff7bda87108cb51adc5f7db4aa367b6e#2021-09-2415:35wilkerlucioon a first sight, I think this is more complicated than that, it has to do with the way the planner works, you can’t have the same attribute twice, that get’s collapsed in a single call, this could change in the future, but for now its a limitation#2021-09-2614:29markaddleman@wilkerlucio fyi https://github.com/wilkerlucio/pathom3/issues/97#2021-09-2707:41Jakub Holý (HolyJak)Hi! A number of links on the net refer to https://blog.wsscode.com/ but it says just 404 There isn't a GitHub Pages site here. 😿 IT would be nice to add an redirect or something...#2021-09-2708:27wilkerluciothanks for pointing it out, I have no idea why that stopped working, github seems to just lost the pages configuration, its up again now šŸ‘#2021-09-2811:36markaddlemanHi @wilkerlucio FYI discussion of Issue 97 https://github.com/wilkerlucio/pathom3/discussions/99#2021-09-2820:51lilactownis there a live demo of pathom viz? i.e. with the youtube or github API#2021-09-3010:43wilkerlucionot online, the query editor don't have a web version outside Fulcro tools or Pathom Viz (the app), but you can use this setup as a testing point with the app: https://github.com/wilkerlucio/pathom3-graphql/blob/main/test/com/wsscode/pathom3/graphql/test/swapi.clj#2021-09-3010:44wilkerlucioyou also need to add the connector dep, I use as a local alias:
com.wsscode/pathom-viz-connector {:mvn/version "2021.07.15-1"}
#2021-09-3010:44wilkerluciomy alias for local devtools for example:#2021-09-3010:44wilkerlucio
:local/devtools
  {:extra-deps {com.wsscode/pathom-viz-connector {:mvn/version "2021.07.15-1"}
                djblue/portal                    {:mvn/version "0.15.1"}
                faker/faker                      {:mvn/version "0.2.2"}
                com.wsscode/reveal-ext           {:local/root "/Users/wilkerlucio/Development/reveal-extensions"}
                vlaaad/reveal                    {:mvn/version "1.3.199"}}}
#2021-09-3016:19lilactownI'd like something I can send my manager to convince him that pathom is a good idea šŸ™‚#2021-09-3016:20lilactownwe do have an endpoint that is currently using pathom, but it's not very powerful yet and to setup pathom viz we need to be able to send custom headers#2021-09-3016:20lilactownall of this takes time I just haven't had. sounds like there isn't an easy example online. I'll wait until I have time to setup a demo myself then. Thanks!#2021-09-3023:36wilkerlucio@U4YGF4NGM if you use the app with the connector the setup is pretty strait forward, no need to go over the custom headers, you can connect the env directly to the app (it uses http calls on CLJ, using its own server thing, and websockets on CLJS), I love to have a web version at some point, but not a priority for me as of now, glad to help to figure that setup#2021-09-2820:52lilactownI'd like to show some coworkers the autocomplete functionality#2021-09-2921:27lboliveiraHi all! Shouldn’t the 3rd process work?
(pco/defmutation foo [{:keys [b]}]
  {:res b})

(pco/defmutation bar [{:keys [b c]}]
  {:sum (+ b c)})

(def indexes (-> (pci/register
                   [(pbir/single-attr-resolver :a :b inc)
                    (pbir/single-attr-resolver :sum :sum-str str)
                    foo bar])
                 (p.plugin/register pbip/mutation-resolve-params)))

(p.eql/process indexes [`{(foo {:a 1}) [:res]}])
=> {user/foo {:res 2}}
(p.eql/process indexes [`{(bar {:a 1 :c 10}) [:sum-str]}])
=> {user/bar {:sum-str "12"}}
(p.eql/process indexes {:c 10} [`{(bar {:a 1}) [:sum-str]}])
Execution error (ExceptionInfo) at com.wsscode.pathom3.connect.planner/verify-plan!* (planner.cljc:1466).
Pathom can't find a path for the following elements in the query: [:c] at path []
#2021-09-3002:29wilkerluciothe entity input doesn't flow to the mutation params, they are their own world, so this is the expected behavior#2021-09-3004:55cjmurphyOh. I thought that in Pathom3 inputs were going to be 'first class citizens' for mutations. The model I seem to want to understand, and others too, has params and inputs as separate things. And both mutations and resolvers have them in the same way.#2021-09-3010:35wilkerluciomutations are a different dimension and their params don't mix with inputs, it was never the plan to mix them, that said, you can make plugins to mix them if you want, but in Pathom mutations are design to be very self contained (@U067NFWUR made it here: https://gist.github.com/loliveira/5c907135b5a40be747aeb527666b7b7c)#2021-09-3010:37wilkerluciothe mixing of inputs and params, IMO that can lead to confusing situations where a mutation operation would vary based on things not other than the mutation call itself, in Pathom the mutations are made to favor predictability (making sure your mutation call is consistent with just the params you send, a single point to check whats coming in)#2021-09-3012:08cjmurphyYeah I never want to mix params and inputs either. They are separate things. Mutations have params only. But it would be good if they can have inputs as well, the way resolvers have params and inputs. That way mutations can for instance pick up global inputs.#2021-09-3012:14cjmurphyIf Pathom users want their mutations to be predictable then they don't need to specify them to take inputs (just params - the usual case). Sometimes you just want your mutation to see something that you know can be resolved.#2021-09-3012:25wilkerlucio@U0D5RN0S1 that's covered with the pbip/mutation-resolve-params, in that way the ::pco/params will use Pathom to fulfill it, is just the part of bringing the current entity (which is the input) mixed in the game that I dont think is a good idea to mix in#2021-09-3012:36cjmurphyI was just thinking of the defresolver macro having the args [env inputs params] and the defmutation macro having the args [env params inputs]. So keeping params and inputs completely separate. The last arg in both of them being optional. So nothing special.#2021-10-1017:50cjmurphyI'd like to be able to understand why this isn't a good idea.#2021-10-1215:19wilkerluciohello @U0D5RN0S1, I'm really not looking forward to change arities on resolvers and mutations (a lot of things depend on this arity, making more would complicate everything), you can make your own macro to support those if you want, I already mention the reasons why I don't think using inputs in mutations is a good idea, to be honest I had some thoughs about this in the past, but never had the time to explore, so I wont put something at library level that I'm not comfortable with. also, remember you can explore and play with this idea yourself by creating some plugins and/or custom macros (to support he arities you are looking for). I'll be happy to learn more about how it goes in your experience, but not looking for this changes ta the library at this point#2021-10-1215:32cjmurphyOkay thanks. I guess I never really understood those reasons. A little bit of documentation might really help the confusion that I think exists out there, although hopefully Pathom 3 makes it all a bit more obvious. For example in the code I'm maintaining at the moment the mutations have inputs declared!! From Fulcro you can often just have what might be inputs in app state, and deliver them as mutation params. The problem comes more with session attributes that feed easily into resolvers, and then why not mutations as well?? Back to the documentation plea - if your audience know why inputs are not supported for mutations, then they won't have any trouble remembering that they're not supported! Thanks for all your great work by the way šŸ˜„#2021-10-1216:00wilkerlucioand I like to say I really appreciate you being around for so long, I still remember when we had a chat during a Clojure conference at lunch time around those, very early days šŸ™‚#2021-10-1216:00wilkerlucioI'm all in for improved docs to make these things more clear, do you have a suggestion on that?#2021-10-1216:34cjmurphyI've never been to a Clojure conference, but I've certainly been around Fulcro for a long time. Docs are hard and what's worse is lots of people don't read them. I've recently read the XTDB docs - they were good b/c they told a story - so much better than the Datomic docs! Your SpaceX talk was great from that point of view (there's another talk out there where s/one actually copies your talk ha!). That's why the Brave Clojure book does so well. Say telling a story setting up a logistics application, repairing Pathom code badly done that has bugs that need to be solved along the way, that shows how much can be done w/out plugins and macros. You did ask!#2021-10-0414:36markaddlemanI have a resolver with output [:x :y :z]. If the resolver cannot compute the result, should it return nil or {:x nil :y nil :z nil} ? I'm asking because, currently, my resolver is returning nil and that causes an Insufficient data calling resolver... error deeper in the plan. However, if the resolver returns {:x nil :y nil :z nil}, the plan is executed successfully. If it's important, I'm running in strict mode. If this is expected behavior, that's fine but a little inconvenient. If it's not expected, I can put together a repro case.#2021-10-0414:42dehliit depends on how you want downstream resolvers to behave. If you wanted to execute downstream resolvers w/ nil values then you should return {:x nil :y nil :z nil} If you don’t want them to execute unless there’s a value, then return nil (and pathom won’t be able to execute downstream resolvers that take x, y, or z as input).#2021-10-0414:47markaddlemanI have multiple resolvers that output [:x :y :y] . I want the next resolver in the OR to run. I think that means returning nil#2021-10-0414:50dehliyep! i think that’s what you want to do#2021-10-0414:54markaddlemanThanks. I'll double check that other resolvers get a chance to run#2021-10-0414:57cyppanIn my understanding Pathom doesn’t care about values, only keys, so if you return {:x nil} or {:x :something} it’s the same for its graph plan, the dependent resolver taking :x as an input will be executed in both cases. But if you return nil without the keys (which is fine, it’s just different semantics) you have to explicitly handle optionality on the resolver which depend on this input with (pco/? :x). Otherwise, in strict mode, the graph plan step will fail to find the path.#2021-10-0414:58markaddlemanThanks!#2021-10-0415:53wilkerlucio@U2845S9KL if you return nil, pathom considers that key realized, and wont try to call other resolvers to fulfill it, in case of not a key, then pathom will try the next OR option#2021-10-0415:54wilkerlucio@U0CL38MU1 you semantic description is on spot#2021-10-0417:02markaddlemanIt turns out I had a resolver with an incorrectly specified output. It declares that it returned :x and :y. It successfully ran before the :x :y :z resolver which caused Pathom to go through a quite a journey to try to obtain :z. I feel like there is a bug in Pathom's plan execution because I think it should have eventually back tracked to find the :x :y :z resolver.#2021-10-0417:03markaddlemanI'll try and form a repro case. More likely, I'll end up discovering more badly specified resolvers :)#2021-10-0417:05markaddlemanOne thing that would have helped me track this down earlier: when Pathom fails to generate a plan or encounters a problem during execution, it would be helpful if it could generate a plan anyway (or the plan it has computed so far) so the plan can be displayed in the Visualizer#2021-10-0418:33wilkerlucioyeah, I agree we need more details on the failure case to help debugging, would love to collect a few examples cases like this, so we can start working a better reporting from those examples#2021-10-0418:57markaddlemanCool. I'll provide some repro cases#2021-10-1013:02dehliFor pathom3, should we use pco/? to declare optional outputs or is that tool just used for inputs? I noticed that i could use it for p.a.eql/process as well if I wanted to query for a keyword that might not come back.#2021-10-1013:38wilkerluciojust for inputs, outputs are kind optional by default#2021-10-1013:39wilkerluciowhat you send to process is analogous to an external input :)#2021-10-1015:01dehlivery cool! thanks for the help as always!#2021-10-1018:22Drew Verleeis pathom viz pathom3 compatible?#2021-10-1021:48wilkerlucioyes, you need to use the pathom3 namespaces on the connector#2021-10-1121:39markaddleman@wilkerlucio Here's an example of an validation check that could be more useful:
Execution error (ExceptionInfo) at com.wsscode.pathom3.connect.runner/check-entity-requires! (runner.cljc:733).
Required attributes missing: [:workflows.workflow/workflow-ids] at path []
Which resolver produced the incomplete output?
#2021-10-1215:23wilkerluciohello Mark, hard to say without having the full context, to collab on that I suggest the following format: 1. here is the full code 2. here is the error message 3. here is how would be nice to have that message#2021-10-1216:16markaddlemanUnderstood. I was rushing a bit when I dropped you that message. I'll provide a complete repro case#2021-10-1218:03kendall.buchananDoes anyone know if there’s a performance trade-off between the following two resolvers, in Pathom3?#2021-10-1218:03kendall.buchanan
(pco/defresolver user-ids-by-group-resolver
  [{db :db}
   {group-id :group/id}]
  {::pco/output [:group.user/ids]}
  {:group.user/ids (expensive-db-query group-id)})
#2021-10-1218:03kendall.buchanan
(pco/defresolver user-ids-by-group-resolver
  [{db :db}
   {group-id :group/id}]
  {:group.user/ids (expensive-db-query group-id)})
#2021-10-1218:04kendall.buchananIn the former, ::pco/output can signal the return value of the resolver without invoking expensive-db-query. Or does macro magic make the two queries identical?#2021-10-1218:46souenzzoboth identical. You can check with macroexpand or do a simple benchmark#2021-10-1221:45wilkerluciothe output will be computed at code read time in this case, so if the result is the same, the runtime is also the same#2021-10-1402:51lsenjovUsing pathom3, is there a way to have a mutation join differently? Use case: having an update-user mutation, that we can send multiple updates through with separately, and see the result of each So:
[('update-user {:id 1 :active true})
 ('update-user {:id 2 :active false})]
then get a response for each
{'(update-user 1) :ok '(update-user 2) :error}
Currently it returns a single entry from whatever came last
{'update-user :error}
#2021-10-1415:27souenzzo@lsenjov in pathom2 you can, if you can't in pathom3, it may be a issue in pathom2 it will aways have this naming issues. as both mutations have the same name, the second will overwrite the result of the first. But talking in a graph-api perspective, I would say that it is a anti-pattern. I would do something like [(update-many-u\ers {:users [{:id 1...} ...]})], then you can return something like {update-many-users {:deleted [{:id 1... } ..]}}#2021-10-1422:26lsenjovFigured that's how it needed to get done, had to check. Cheers#2021-10-1700:01wilkerluciohello folks, its on master! parallel processing on Pathom 3 šŸ˜„ This is a big milestone and I'm very excited about it! I'm gonna play for a few days with it before cutting a release, but anyone wanting to try its on master now! To play with it, you should use the async runner (from [com.wsscode.pathom3.interface.async.eql :as p.a.eql]) and add ::p.a.eql/parallel? true to your env, and you get the parallel runner!#2021-10-1700:12wilkerlucioOne important thing to note, for things to go parallel you need resolvers that do async operations, if all resolvers are sync you wont get the benefit, the simplest way to make some expensive operation async is to wrap it with (p/future ...) to make it run in another thread but in CLJS this is natural, given IO operations are always async anyway#2021-10-1816:42dehliThis is very cool! We’re playing around with it in our app and so far it’s working great.#2021-10-1702:52wilkerlucioNew docs for parallel process in the website: https://pathom3.wsscode.com/docs/async/#parallel-process#2021-10-1814:19mauricio.szaboHi there! I'm having trouble figuring out if I can handle a case of a resolver returning multiple inputs. So, the idea is: I have a resolver that supposedly return :user/login and :user/password for example. I want it to return an array - is it possible? Instead of returning a map with both attributes, return a vector of maps?#2021-10-1814:22wilkerlucioyes, taht's correct, give it a name and return a vector of maps#2021-10-1814:23wilkerlucioQuery looking something like this:
[{:user/find-logins [:user/login :user/password]}]
#2021-10-1814:27mauricio.szaboThanks, I'll try šŸ™‚#2021-10-1819:41dehliFrom within the process-resolver-error or process-mutation-error plugin can I somehow assoc in an extra key into the response if a certain condition is met? I want to trigger a different codepath post processing if a certain error is triggered and I’m trying to figure out the best way to do this (note I’m using lenient mode) My other thought is that I could iterate through the entire response post-processing and search for the error but this doesn’t seem as clean due to the fact that errors can exist at various paths in the response.#2021-10-1820:06dehliI think I’m going to go the postwalk approach instead, but if there’s a better way I’m all ears šŸ™‚#2021-10-1820:12dehliactually postwalk ended up not being too bad! just had to find instances of ExceptionInfo#2021-10-1820:42wilkerluciothere is also the plugin entry point ::pcr/wrap-resolver-error that you can use#2021-10-1820:43wilkerluciocheck this: https://github.com/wilkerlucio/pathom3/blob/13b1ae92b244e136871231d650e6f2ff667a0f74/src/main/com/wsscode/pathom3/connect/runner.cljc#L302-L317#2021-10-1820:43wilkerluciomaybe you can do something around that#2021-10-1820:43wilkerluciowould love to hear if that works, or if it doesn't, why not#2021-10-1820:54dehlioh great! i will give that a shot as well. appreciate it!#2021-10-1922:10wilkerlucio#2021-10-2003:14wilkerlucioin another set of good news, I did the first OR node optimization today! this optimization covers cases where sub-branches of OR nodes are sub-paths of each other (same resolver sequence, but one with extras), here are screenshots of a before/after the optimization, this is on master but not released yet#2021-10-2003:15wilkerluciothe example in case is playing with the Twitter API, there are some user data that comes strait from the tweet endpoint, but the tweet endpoint has a reference for the user-id, which leads to a secondary option to fetch the user data form the user endpoint#2021-10-2003:15wilkerluciothis is what generates that OR case (get from tweet directly vs get user id from tweet then get data from the user endpoint)#2021-10-2003:16wilkerlucionote that in this case, the most common result is that the tweet endpoint will get all the data already, and the resolver to call the user endpoint will be skipped, given the data will be available already#2021-10-2003:17wilkerlucioas you can see in the ran version of the graph#2021-10-2003:18wilkerluciobut, if for some reason the tweet endpoint misses some of the user data, then the user data resolver will get called for a second chance to get the data#2021-10-2013:45mauricio.szaboWow, I think I'll be able to make good use of this optimization!#2021-10-2214:21dehliGood morning! Is this behavior expected in pathom3 (where the same keyword returns different values based on other keys you ask for)? I can see that it’s b/c the :node/index key takes :node/parent as its input but it’s unexpected (to me) that those extra keys would leak through to the response. https://github.com/dehli/pathom-edge-cases/blob/e3cc7d39db4ee10aa654d018d65193adce06b6de/pathom3/extra-keys/main/src/dev/dehli/pathom3/extra_keys.clj#L33-L45#2021-10-2215:43wilkerluciothis is expected because you are not specifying the details of :node/parent, so it gets everything#2021-10-2215:43wilkerlucioI think you may want to use a recursive query there, in which case it would liimt the results#2021-10-2215:44wilkerluciolike:
(p.eql/process env {:node/id 1} [:node/id {:node/parent ['...]}])
#2021-10-2215:59dehliokay, that makes sense. i’ll need to think through how to restrict asking for :node/parent without sub keys since best practice would be to always specify all the requested keys (so that internal values won’t get returned)#2021-10-2215:59dehlithanks!#2021-10-2221:27Drew VerleeIn Phatom3, is there a way to see all the attributes reachable given a input? I'm using PathomViz and the index explorer > provides view only seems to show the next node.#2021-10-2223:56Reily SiegelTry com.wsscode.pathom3.connect.indexes/reachable-paths#2021-10-2300:37wilkerlucioI recently noted a bug on the attribute indexing, it may be it, its related to nested atrributes, they are not getting pulled out#2021-10-2300:38wilkerluciobut also on the UI, there is the depth choice, by default the graph view only shows 1 depth#2021-10-2300:38wilkerluciobut you can click to increase#2021-10-2300:39wilkerluciocan you make a small case of what you see/miss? just wanna make sure its the same thing or a possible new issue#2021-10-2302:36mauricio.szaboIn Pathom3, is there a way to not capture the errors? I mean, if something fails, throw the exception instead of capturing and wrapping it on the output?#2021-10-2320:11wilkerluciothe strict mode works like that#2021-10-2400:00mauricio.szaboIn strict mode, if an exception happens, does it bubbles to the eql call?#2021-10-2406:10wilkerlucioyes#2021-10-2319:32roklenarcicI wonder how ::pc/input works. If I have query like: [:user/id {:user/accounts [:acc/id {:acc/domains [:acc/emails …. If I create a resolver ::pc/input #{:user/id} ::pc/output [:acc/emails] would it work? I guess the question is what are the available inputs for a resolver at a certain join… just the data at the same level?#2021-10-2320:09wilkerluciojust at same level, when depth changes pathom considers it a different entity, and atrributes are not automatically shared with parent entity#2021-10-2320:10wilkerluciobut you can send parent data down when creating the relationship, it makes sense to do it in some cases#2021-10-2408:45roklenarcicHow do I do that? By adding one of the inputs to the output?#2021-10-2415:18wilkerluciohumm, I'm taking a second read, seems like your issue is in a different direction than I was though when I read before#2021-10-2415:18wilkerlucioin this case there is a possible ambiguity issue to be aware#2021-10-2415:19wilkerluciobecause you have multiple accounts, but wnat the data from a specific account at same level of the user#2021-10-2415:19wilkerlucioyou can make resolvers that pull from the inner value to the outer (better with Pathom 3 since it supports nested inputs, which Pathom 2 doesn't)#2021-10-2516:00kendall.buchanan@wilkerlucio Would you mind elaborating on mutations? I’m struggling to see what value they offer when it takes a direct reference to the mutation var to initiate one. How is that any better, or worse, than invoking a function directly?#2021-10-2516:01wilkerluciomutations are the "call side" of EQL, if you are not exposing the graph with some external EQL (meanding you are just using pathom internally), you could replace most of it with a regular fn call.#2021-10-2516:02wilkerlucioanother difference is that mutation responses are also integrated with Pathom, so a mutation may respond with just :user/id, and the requester will call the mutation and ask for extra data, like: [{(create-user {...}) [:user/name :user/email]}]#2021-10-2516:03wilkerlucioin Pathom 3 there is another interesting thing for mutations, that is to auto-resolve the mutation params, in this case you can use Pathom engine to help materialize the params using the resolvers, for example, if a mutation requires :user/full-name, but the user invokes it with {:user/first-name "foo" :user/last-name "bar"}, Pathom can process the :user/full-name and send the params in this way#2021-10-2516:03wilkerluciomakes sense?#2021-10-2516:32kendall.buchananYeah, yeah, that’s super helpful…#2021-10-2516:32kendall.buchananAlthough, I was under the impression that auto-resolving of params wasn’t handled automatically…#2021-10-2516:33kendall.buchananAh, mutation-resolve-params#2021-10-2516:33kendall.buchananGot it.#2021-10-2516:33kendall.buchananK, that’s really helpful, thank you.#2021-10-2516:47wilkerluciohappy to hear feedback on that, this feature didn't got so much usage yet, one of the things that's not clear is what should happen if the params can't be fulfilled, for now it will throw, but I'm willing to change depending on the discovery of new use cases#2021-10-2516:55kendall.buchananI don’t care for the syntax around invoking them, namely having to include the namespaces.#2021-10-2516:55kendall.buchananI wonder if the same goal could be achieved through a namespaced keyword.#2021-10-2516:56kendall.buchananOr dealing with quoted function names.#2021-10-2518:08fjolne> it takes a direct reference to the mutation var to initiate one @U0HJA5ZQT if I understood you correctly it's not the case: mutations are referenced by (typically fully-qualified) symbols, calling of mutations defined by pco/defmutation is just a convenience, e.g. [( {:x 1})] is essentially the same as [(https://my.app/mutate ~{:x 1})]`#2021-10-2519:39wilkerlucio@UDQ2UEPMY what you described is the Fulcro behavior around mutations, Pathom 3 is different in terms of calling it, in Fulcro, when you call a mutation, it returns a reference to the call (so you can kind transparently do it), in Pathom 3 this will actually invoke the mutation, so doing [( {})], is going to call the mutation and put a map on the query, which is usually not what you want#2021-10-2519:39wilkerluciomutations need to be quoted because they are, in principle, a "call over the wire"#2021-10-2519:39wilkerlucioso its about sending a command to be executed by the processor
#2021-10-2519:40wilkerlucio@U0HJA5ZQT mutations are symbols to separate their semantics from the semantics of attributes, mutations are commands, attributes are data points#2021-10-2519:44kendall.buchanan@wilkerlucio: Yes, it’s just that with regular resolvers, the namespace in which you make a query is decoupled from the ultimate source of the data. You don’t have to know where the data is to get it. With mutations, there’s at least a sense of having to know ā€œwhereā€ the mutation resides. Feels jarring in contrast, is my only point.#2021-10-2519:44kendall.buchanan(And has one additional point of coupling—that of a namespace.)#2021-10-2519:47wilkerlucioare you meaning about the name of the mutation? you can override that, for example:
(pco/defmutation create-todo [...]
  {::pco/op-name 'acme.todos/create}
  ...)
#2021-10-2519:48wilkerlucionow, this mutation will be invoked by ['(acme.todos/create {...})]#2021-10-2519:52kendall.buchananMmm, gotcha. Okay, thx.#2021-10-2519:53wilkerlucioin Pathom 2 that's also available:
(pc/defmutation create-todo [...]
  {::pc/sym 'acme.todos/create}
  ...)
#2021-11-2216:03kendall.buchanan@wilkerlucio: Coming back to mutations… You note in the documentation that resolving params in mutations has a performance hit (naturally). Is that hit large?#2021-11-2216:04kendall.buchananAlso, is there a simple way to make that choice on a per-mutation basis?#2021-11-2216:47wilkerlucioits the same cost as if you ran the query by yourself inside of it, so it depends on how complex the resolution is#2021-11-2216:48wilkerluciono way to set per mutation, but thats easy to make, a bit of change on the plugin code we can check for some flag in the mutation config#2021-11-2217:44kendall.buchananOkay, thx! I’ll see how it goes turning it on completely.#2021-11-2219:10kendall.buchananSorry to bug you with this again, but is this expected behavior?#2021-11-2219:10kendall.buchanan#2021-11-2219:10kendall.buchanan#2021-11-2219:11kendall.buchananIt appears that upon enabling the plugin only keys explicitly declared in params are available. That seems to prevent the possibility of using optional keys, unless added to a grab bag key.#2021-11-2220:36wilkerlucioyes, its expected, because it becomes a query like running from eql/process, but I see it might need better documentation#2021-11-2220:36wilkerlucioI encourage you to copy the code from the plugin and adjust for your needs in case you want a different behavior#2021-11-2220:45wilkerlucioand no worries about bringing those, love to get those feedbacks šŸ™#2021-11-2222:08kendall.buchananCool, it’s good for now… I’m just adding those keys one level deeper.#2021-10-2616:04Matthew ThompsonHi everyone! In Pathom3, is there a way to use wrap-resolver-error or similar to replace an exception thrown in a resolver with a custom error key in the returned output from the resolver? I just want to be able to log errors and put an error key of my own in the returned results.#2021-10-2616:46wilkerlucioit should be possible, yes, if not please bring a case and we can look together#2021-10-2616:47wilkerlucioanother option is to use the Lenient Mode, which never throws#2021-10-2709:10Matthew ThompsonThe issue I have is that with lenient mode on, instead of throwing the exception, the exception message and stacktrace gets returned in the output of the resolver. This causes issues later when attempting to marshall this output into transit. I'd just like to replace the exception in the output with a key so i know there was an exception.#2021-10-2815:05donavanI think what we would like is for something like ::pcr/wrap-resolver-error to be able to resolve the node with a custom error value, does that make sense Wilker?#2021-10-2815:25wilkerlucioyou can mutate that in the stats, like the mark-node-error does (the wrap-resolver-error wraps around this function): https://github.com/wilkerlucio/pathom3/blob/85a2b0596f9798d97087d301e0579e3e0abc52cb/src/main/com/wsscode/pathom3/connect/runner.cljc#L309#2021-10-2815:25wilkerlucio(or throw directly, if that's the case)#2021-10-2616:48wilkerluciohello everybody, just a reminder, in ~ 40 min I'll be presenting at London Clojurians about Pathom 3! if you have the time I would love to see you there! https://clojurians.slack.com/archives/C03RZRRMP/p1634644636108900#2021-10-2619:07pithylessThanks for the presentation @wilkerlucio! And for all the ongoing work in Pathom3.#2021-10-2619:08pithylessP.S. Demo gods weren't particularly angry; I'd say all things considered - Great Success! :)#2021-10-2619:19wilkerluciohahha, happy it worked after all, now that I'm thinking about it, the problem was that the number was bigger than the JS supports, and was probably lost precision on the wire or something#2021-10-2619:19wilkerluciowhile the string doesn't have this issue (and worked šŸ™‚)#2021-10-2619:44wilkerluciosource code for presentation: https://github.com/wilkerlucio/presentation-data-navigation-with-pathom3 slides: https://www.dropbox.com/s/yyjvbrcipkthzh3/data%20navigation%20with%20pathom%203.pdf?dl=0#2021-10-2714:59Piotr Roterskivideo: https://www.youtube.com/watch?v=YaHiff2vZ_o#2021-10-2619:09Jakub Holý (HolyJak)Hi! Why I query for a nested entity attribute without a join such as :family/members, I get by default all the data (i.e. the name, id, ... of each member). On the other hand, if I query for an entity by ident (such as [:person/id 1]) I do not get back all its attributes by default but only the ident itself. Why is that?#2021-10-2619:18wilkerlucioits because idents are just a single attribute provider, when you use though a link, that resolver is already providing some data (and that's what you get when you don't asked for the details of it), but the ident will just give a single attribute, and wont call any resolver#2021-10-2621:51Jakub Holý (HolyJak)Thank you!#2021-10-2619:44wilkerluciosource code for presentation: https://github.com/wilkerlucio/presentation-data-navigation-with-pathom3 slides: https://www.dropbox.com/s/yyjvbrcipkthzh3/data%20navigation%20with%20pathom%203.pdf?dl=0#2021-10-2621:01mauricio.szaboHi @wilkerlucio - I think I found a bug on Pathom3, batch resolvers, and params. I opened an issue here, please let me know if I can help in some way to fix it: https://github.com/wilkerlucio/pathom3/issues/105#2021-10-2714:59Piotr Roterskivideo: https://www.youtube.com/watch?v=YaHiff2vZ_o#2021-10-2816:33jgertmIs this a bug, or WAI?
(def env
  (pci/register
   [(pbir/static-table-resolver :module/id
                                {"foo" {:ast/definitions [{:name "bar"}
                                                          {:name "baz"}]}})
    (pco/resolver `annotate-ids
                  {::pco/input [:module/id {:ast/definitions [:name]}]
                   ::pco/output [ {:ast/definitions [:ast/id]}]}
                  (fn [env {:keys [module/id ast/definitions]}]
                    {:ast/definitions
                     (mapv
                      (fn [{:keys [name]}] {:ast/id [id name]})
                      definitions)}))]))

(try
  (p.eql/process env
                {:module/id "foo"}
                [{:ast/definitions [:ast/id]}])
  (catch Throwable t t))
;; => #error {
 :cause "Pathom can't find a path for the following elements in the query: [:ast/id] at path [:ast/definitions 0]"
 :data {:com.wsscode.pathom3.connect.planner/graph #:com.wsscode.pathom3.connect.planner{:nodes {}, :index-ast #:ast{:id {:type :prop, :dispatch-key :ast/id, :key :ast/id}}, :source-ast {:type :join, :dispatch-key :ast/definitions, :key :ast/definitions, :query [:ast/id], :children [{:type :prop, :dispatch-key :ast/id, :key :ast/id}]}, :available-data {:name {}}, :unreachable-paths #:ast{:id {}}}, :com.wsscode.pathom3.connect.planner/unreachable-paths #:ast{:id {}}, :com.wsscode.pathom3.path/path [:ast/definitions 0], :com.wsscode.pathom3.error/phase :com.wsscode.pathom3.connect.planner/plan, :com.wsscode.pathom3.error/cause :com.wsscode.pathom3.error/attribute-unreachable}
 :via
 [{:type clojure.lang.ExceptionInfo
   :message "Pathom can't find a path for the following elements in the query: [:ast/id] at path [:ast/definitions 0]"
   :data {:com.wsscode.pathom3.connect.planner/graph #:com.wsscode.pathom3.connect.planner{:nodes {}, :index-ast #:ast{:id {:type :prop, :dispatch-key :ast/id, :key :ast/id}}, :source-ast {:type :join, :dispatch-key :ast/definitions, :key :ast/definitions, :query [:ast/id], :children [{:type :prop, :dispatch-key :ast/id, :key :ast/id}]}, :available-data {:name {}}, :unreachable-paths #:ast{:id {}}}, :com.wsscode.pathom3.connect.planner/unreachable-paths #:ast{:id {}}, :com.wsscode.pathom3.path/path [:ast/definitions 0], :com.wsscode.pathom3.error/phase :com.wsscode.pathom3.connect.planner/plan, :com.wsscode.pathom3.error/cause :com.wsscode.pathom3.error/attribute-unreachable}
   :at [com.wsscode.pathom3.connect.planner$verify_plan_BANG__STAR_ invokeStatic "planner.cljc" 1490]}]
 :trace
 [[com.wsscode.pathom3.connect.planner$verify_plan_BANG__STAR_ invokeStatic "planner.cljc" 1490]
  [com.wsscode.pathom3.connect.planner$verify_plan_BANG__STAR_ invoke "planner.cljc" 1480]
  [com.wsscode.pathom3.connect.planner$verify_plan_BANG_ invokeStatic "planner.cljc" 1509]
  ...]}
#2021-10-2816:37wilkerluciothe problem is that you are trying to redefine the value of :ast/definitions, you cannot change a value once its defined (and its gets defined when you use it as the input)#2021-10-2816:38wilkerlucioyou can note your resolver isn't even called, because :ast/definitions is realized by the static table resolver#2021-10-2816:38wilkerluciousing a different name will work#2021-10-2816:38wilkerlucio
(def env
  (pci/register
    [(pbir/static-table-resolver :module/id
       {"foo" {:ast/definitions [{:name "bar"}
                                 {:name "baz"}]}})
     (pco/resolver `annotate-ids
       {::pco/input [:module/id {:ast/definitions [:name]}]
        ::pco/output [{:ast/definitions-id [:ast/id]}]}
       (fn [env {:keys [module/id ast/definitions]}]
         {:ast/definitions-id
          (mapv
            (fn [{:keys [name]}]
              {:ast/id [id name]})
            definitions)}))]))

(comment

  (p.eql/process env
    {:module/id "foo"}
    [{:ast/definitions-id [:ast/id]}]))
#2021-10-2816:42jgertmI see, so as a general rule, for a given entity, all attributes need to be defined in the same resolver?#2021-10-2816:45wilkerlucioI wouldn't say that, because you can always extend sideways from resolvers, but you can't get an input and replace it with modified data, once an attribute is set, it doesn't change anymore#2021-10-2816:45wilkerluciothis is important to keep consistency across the process#2021-10-2816:45wilkerlucioin your case you can see you ask for :ast/definition on the input, and try to set it modified on the output, which can't happen#2021-10-2816:46wilkerlucioideally in your case, I think :ast/definitions should have the ID already, instead of trying to compute it based on the :module/id#2021-10-2816:47wilkerlucio
(pbir/static-table-resolver :module/id
  {"foo" {:ast/definitions [{:id ["foo" "bar"]}
                            {:id ["foo" "baz"]}]}})
#2021-10-2816:47wilkerlucioso then you can have a resolver to extract the :name from the :id, altough you should use namespaced keywords there, to avoid making definitions in names that are too broad (like :id and :name)#2021-10-2816:52jgertmUnderstood. Note that this is a simplified example, in my actual code I have a parser that produces the AST, without node IDs, and I figured I would write a resolver that adds them, but maybe that's too odd because node IDs have their parent's ID as a prefix.#2021-10-2817:26jgertm@U066U8JQJ would you say that Pathom3 is actually a good fit for writing a query-based compiler?#2021-10-2914:04wilkerlucio@U0YAC4ZMZ sorry, I don't have enough context on this problem to make a statement, can you tell more about the case?#2021-10-3015:42lilactownI haven't tried out pathom 3 yet. the talk you gave @wilkerlucio made the remote resolver look awesome. if I'm exposing a pathom API over HTTP, is there anything different that my consumers will need to do if/when I move to pathom 3?#2021-10-3015:52wilkerluciothat part of the external interface is fully compatible, ita about changing the engine, but the interface for clients remains the same, pathom3 does extend it tought, on top of the eql request (that takes the vector form) there is also the new map format, the intention is to allow more than just the query, like data for the root entity#2021-10-3015:53wilkerlucio(the map format is provide by what I call boundary interface in pathom3)#2021-10-3015:57lilactownthat's great. i'll look in the docs for the map format, as i'll want to at least think about how to add support for it later#2021-10-3015:44lilactownI understand the implementation of the parser et al. is very different. we are making use of pathom2 now and starting to build some client-side tools around it and I'd like to ensure those tools are future proof. atm they basically boil down to "submit an EQL query via POST" and it seems like it wouldn't be different with pathom3#2021-10-3015:56robert-stuttafordhey @wilkerlucio šŸ™‚ a SUPER common situation we have is we're pulling data out of the db on the server, and then shipping that plus some env to the client, and then composing + deriving a bunch of stuff for a react app to render / make interactable. it seems that we could rewrite a lot of this as resolvers in cljc, and then somehow use them on both sides of the wire - so that when we need that composition / derivation on the server, it's there, but when we use it on the client, it fetches the stuff that only the server can provide, using the distributed computing mode you demonstrated. that's Thing One. Thing Two is, wrapping the server's impl in an access control layer. if you're allowed, you get the data, but if you're not, you get an error. secondly, if you're allowed, the data you get is based on who you are (think social graph). is there perhaps a working example somewhere, showing pathom3 on both the client and the server, in cljs + clj respectively, doing that distributed example in your London talk, on which i could experiment to get a spike of all of this going?#2021-10-3018:42lilactown@U0509NKGK i’m curious: do you (and also looking for other people) often run pathom on both the client and server?#2021-10-3018:42lilactownI’ve been approaching it as a purely server-side tech, submitting queries via HTTP#2021-10-3018:43lilactownin my head it seems like pathom on the client would be a lot of additional overhead but maybe I just don’t understand how the client-server integration could work#2021-10-3019:30robert-stuttafordi've never used pathom before#2021-10-3019:31robert-stuttafordjust thinking about how i'd like to try to use it!#2021-10-3019:35lilactownaha šŸ˜„#2021-10-3019:36lilactownwe basically use it like GraphQL atm. send queries get data back. cache them in a normalized store#2021-10-3118:16wilkerlucioI'm planning to write such demo soon, its quite similar to the one I did on the presentation, the major difference is to also make the proper setup on the CLJS side (regading transit encoding)#2021-10-3118:17wilkerlucioone other architecture I see is also adding web workers to this game, so you may have 3 pathom systems going on, one at the UI level, one at worker level and one at server level#2021-10-3118:28robert-stuttaford@wilkerlucio i'm pretty keen to get this working so if i can assist, lemme know#2021-10-3118:29wilkerluciothanks, that's very welcome, specially the testing part, this is a part of the pathom with little exercising at this time, so I consider experimental stage from it, but the best way I see to get it to a reliable solution is to try it out and find the rough edges šŸ™‚#2021-10-3118:30robert-stuttafordi'm also a Datomic on-prem user, so keen to do this with your datomic spike too šŸ™‚#2021-10-3118:34wilkerluciofor the datomic, altough there is a going on attempt (https://github.com/wilkerlucio/pathom3-datomic) to make everything as automated as possible, IME there is also the problem of controlling/security, unless you are writing a database manager (or something else supposed to have full access) it seems like having specific smaller implementations (in specific resolvers) is better to handle environments with tighter access control needs#2021-10-3118:36robert-stuttafordyep. a large part of our domain deals directly with this aspect of things. i'd really love to build out a set of resolvers that model this on top of what you're doing there#2021-10-3015:57robert-stuttafordi can see P3 removing tons of code here šŸ˜„#2021-10-3015:59robert-stuttaford(seems my questions are similar to lilac's)#2021-10-3109:25robert-stuttafordis there a way to filter out stuff that goes to PathomViz, like Datomic conns and dbs? turns out they OOM when transit tries to emit them šŸ˜† for now i'm just treating them as globals, but it'd be nice to have a conn and a db resolver instead#2021-10-3118:13wilkerluciohello Robert, currently there is no option to do that for a specific location, but I feel your pain, some payloads are just huge (and other non encodable) and that cripples the tooling#2021-10-3118:14wilkerlucioone option you have today is to ask only for the shape of the outputs, instead of the data, setting the flag ::pcr/omit-run-stats-resolver-io? true, this will make the output contains the shapes of the data, but not the data#2021-10-3118:15wilkerlucioI'm wondering what a good way would be to fix this, I have two ideas in mind#2021-10-3118:15wilkerlucio1. make it per-resolver, and mark that a specific resolver shouldn't log its outputs#2021-10-3118:15wilkerlucio2. make it per-attribute, so the user can say some specific attributes should not log their values (in both inputs and outputs)#2021-10-3118:16wilkerlucioI'm more inclined to do solution 2, seems more general and more reusable#2021-10-3118:28robert-stuttafordyeah, 2 would be great#2021-10-3118:29robert-stuttafordor if we could run the payload through a function just before it sends, we could alter or control it as we like#2021-10-3118:29robert-stuttafordi'd simply #(dissoc % :datomic/conn :db) but i could imagine running transformations to summarise/aggregate purely for ease of Viz#2021-10-3118:31robert-stuttafordViz is fantastic by the way#2021-10-3118:34robert-stuttaford@U066U8JQJ would you be open to hanging out for 30-45 mins some time to map out the work here, so we can figure out how i can help? šŸ™‚#2021-11-0113:44mynomoto@robert-stuttaford I wonder if creating a resolver for the datomic db is not a security risk.#2021-11-0113:58robert-stuttafordit is, if you expose pathom directly to the network, yeah!#2021-11-0114:01mynomotoI thought that this is how it should be used šŸ˜… Thanks!#2021-11-0114:43wilkerluciothere is an example on the plugins page on how to prevent certain attributes to getting out: https://pathom3.wsscode.com/docs/plugins#the-wrapping-model-of-plugins#2021-11-0116:28robert-stuttafordthis is great!#2021-10-3115:47souenzzo@robert-stuttaford, yes, you are using pathom2 or 3? high level how: You will need to reimplement/overwrite this resolver, that returns the index https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/connect.cljc#L2079#2021-10-3116:34robert-stuttafordthanks @U2J4FRT2T! pathom3. i'll track it down#2021-10-3118:18wilkerlucioI don't recommend trying this on Pathom 2 though, the implementation there is old, naive and its not getting upgraded#2021-10-3118:27robert-stuttaford3 all the way here šŸ™‚#2021-10-3118:42robert-stuttafordam i nuts, or could i set up multiple indices and compose them, i.e. resolvers using smart-maps from layers lower down?#2021-10-3118:52wilkerlucioI think so, can you make an example to be more specific on what you have in mind?#2021-10-3119:14robert-stuttafordtotally#2021-10-3119:15robert-stuttafordsmart-map (aka resolver index) one: general info about the session/user/environment smart-map two: specific info for a specific sub-system, e.g. access control stuff for a CMS Two's resolvers have access to One's smart-map#2021-10-3119:16robert-stuttafordthat should work right, there's nothing preventing this sort of composition#2021-10-3119:33wilkerlucioI mean a code example šŸ˜…, I'm trying to understand exactly how to mean to integrate them, to rationalize about the implications#2021-10-3119:38wilkerluciosmart maps as values I don't think will work as expected, because there is a lot of merging going on and smart maps merge won't be mergning their indexes#2021-10-3119:39wilkerlucioalso, in general, smart maps are more costly than running EQL queries, and on the client-side they don't support async process#2021-10-3119:40wilkerlucioso, in general EQL is the preferred interface, the use case I see for smart maps are more related to using Pathom to integrate with things that require some kind of map interface, but if you don't have such restriction, EQL should be preferred#2021-10-3120:21robert-stuttafordok great, looks like my question produced value šŸ˜„#2021-11-0112:06Jakub Holý (HolyJak)Hi @wilkerlucio, I really enjoyed your data navigation with P3 talk, thanks a lot! It is both a great explanation of the problems with REST (and GraphQL) and a wow-ing demonstration of the powers of Pathom. Certainly going to recommend it further!#2021-11-0113:48mynomotoIs it possible to update the env from a mutation?#2021-11-0113:49mynomotoUsecase: I write on datomic and want to add the db-after to the env.#2021-11-0113:54pithylessI have the same use-case and pass {:db (atom (datomic/db datomic-conn)), :datomic-conn datomic-conn} as env. That way I can update :db with :db-after during the mutation and use @db in the resolver.#2021-11-0113:58mynomotoDoes the return of the mutation updates the env? I'm not following how what you are suggesting works.#2021-11-0114:00robert-stuttaford@U05476190 do you use PathomViz? if you do, how do you remove the datomic data from your env before sending it to Viz?#2021-11-0114:06pithylessNo, at the moment I'm using P2 and Fulcro inspector (which shows EQL and some perf stats), but not the newest PathomViz which shows env, etc. But I've been following your thread about cleaning up the env data @U0509NKGK and I'm looking forward to a clean approach (and finally getting PathomViz setup and migrating to P3).#2021-11-0114:06pithylessI will say the problem seems similar to issues in P2 with it returning indexes that were not serializable, where we just made a custom resolver middleware that dissoc the pathom-indexes, etc.#2021-11-0114:10pithyless@U05094X3J - in the mutation, you want to (reset! (:datomic-db env) db-after) as a side-effect, and then return whatever you normally return from the mutation; and in the resolver you want to use (deref (:datomic-db env)) as the DB you pass into queries. Since you are storing it as (atom (datomic/db datomic-conn)) the atom takes care of the coordination; and Pathom guarantees read-after-write semantics for "first run mutations, then run resolvers"#2021-11-0114:11pithyless(Going afk now, but hope that makes sense)#2021-11-0114:11robert-stuttafordit does, ta!#2021-11-0114:11mynomotoOh, I missed the atom part. This helps a lot, thanks!#2021-11-0210:04Bjƶrn EbbinghausIn Pathom 2 you can return a new env in your mutations: {:com.wsscode.pathom.core/env (assoc env :db db-after)}#2021-11-0210:20pithylessInteresting… #TIL#2021-11-0212:34mynomotoIs there something like this env update on pathom3 @wilkerlucio ?#2021-11-0218:30wilkerlucioI'm a bit reluctant to add this back, the reason is that I've seen a couple of cases where you end up leaking your env out by accident, and that is really bad, so I hater find another way to handle those cases, the atom is a valid one that's less risky#2021-11-0218:43mynomotoOk, thank you!#2021-11-0411:51souenzzoI recoomend use ::my-app/db as a "input", not as a env#2021-11-0411:22fabraoHello, how do you use clojure testing with resolvers if I want to use interactive testing like in Calva? I tried changing the resolver, evaluated again and it seems keeping the old version#2021-11-0412:43wilkerlucioyou have to remember to also reload the file that builds up the env, so you can rebuild your indexes with the new resolver configuration#2021-11-0413:37fabraoI think the problem is that the resolvers are in def , so I have to reload the namespace and the namespace that call that name space, I did it and it worked, thank you#2021-11-0415:50robert-stuttafordwe can use the #'my-resolver trick when building the index right @U066U8JQJ?#2021-11-0415:55wilkerluciothat is not enough, the problem is that what matters for pathom is the indexes, which get created on the pci/register call, this is why this needs to be re-evaluated when resolver changes to compute the new indexes#2021-11-0415:56wilkerluciothere are possible tricks around, like keeping the indexes in an atom and derrefing at run time (using plugins), this approach has an issue with removing things, another approach would be to keep the resolvers in some mutable atom, and on each call re-compute all of it (something you surely don't wanna do in prod, but is useful in dev time)#2021-11-0416:05jmayaalvwhat we do is to create pathom-env as fn, and during testing/dev we create always a new one on each call. ex:
(defn query! [context tx]
  (interface/query! (p.eql/boundary-interface   (interface/pathom-env context)) txl))
#2021-11-0416:06jmayaalvsimilar for dev, we have an interceptor that creates the pathom boudnary-interface for each request#2021-11-0506:25robert-stuttafordnice#2021-11-0512:17jmayaalvstill the downside is that pathom-viz connector doesn’t see the changes, so when needed we connect to connector via http that triggers the reload interceptor.#2021-11-0419:55cyppanI’m wondering what is the expected behavior of having nested and optional input in a resolver input at the same time (ex: ::pco/input (pco/? {:nested [:id :name]}))? I’ve tried and I don’t get the result I would expect.#2021-11-0421:03cyppanHere what I mean, I think it’s a bug actually https://gist.github.com/cyppan/bbdfdb16d267051ad1e675c4a38ced5f#2021-11-0502:14wilkerluciothat optional is misplaced, it should be [{(pco/? :nested) [:id :name]}]#2021-11-0508:50cyppanoh yes I messed up my repro case sorry I’ve updated the gist, I still have the problem https://gist.github.com/cyppan/bbdfdb16d267051ad1e675c4a38ced5f#2021-11-0822:20wilkerlucioconfirmed the report, tracking issue: https://github.com/wilkerlucio/pathom3/issues/107#2021-11-0909:26cyppanok thanks#2021-11-0519:51cjsauerHey all, has anyone ever run into ā€œrecursive unionā€ queries? I’ve been playing with pyramid which has some EQL support and documented https://github.com/lilactown/pyramid/issues/13, but then realized it is likely a more general EQL problem. How would one handle this in pathom for example?#2021-11-0519:53cjsauerIt seems that both pyramid and pathom’s recursive primitive, namely the '... symbol, assume that the direct parent is the target of that recursion. Is there any way to say that, for example, the grandparent is the target of recursion?#2021-11-0519:54cjsauerOr said another way, how can one recursively query a heterogenous tree with EQL?#2021-11-0520:00cjsauerOkay I actually just https://github.com/lilactown/pyramid/issues/13#issuecomment-962179903 šŸ˜… I was reaching for a union query unnecessarily. Namespaced keywords let one just ask for everything at the root, and then the parent is indeed the proper recursion target.#2021-11-0718:49sheluchin@wilkerlucio if you have a minute sometime can you keep me honest here please? https://clojurians.slack.com/archives/C053AK3F9/p1636299859332600?thread_ts=1636204030.290000&amp;cid=C053AK3F9 Is this an accurate description of the approach you recommend with Pathom? I searched the docs for "qualified keys" and other variations thereof and only found a few small mentions. Are these ideas you mostly explore in the videos linked here: https://github.com/wilkerlucio/pathom/blob/b36c4c494493dc8819b6a89f977306c57d3840d1/README.md#see-a-talks-on-the-concepts or am I reaching in this description?#2021-11-0720:08Nikolas PafitisHello guys, I just started playing around with pathom3 after watching the london clojurians talk. I got a question, how does a resolver that returns a collection of data look like? Is it a resolver that returns a map with a single kv pair and the value's the collection basically?#2021-11-0812:43souenzzoYes. Every "value" in pathom should be "named" "named" values can be requested by a query. If you write a resolver that returns a collection directly, how would the query be?! For example:
;; just a example. It do not work.
(defresolver my-coll [env input]
  [1 2 3])
Which query you need to run to get [1 2 3] back?! It can't fit in the model. To solve this, we always return the value with a keyword
(defresolver my-coll [env input]
  {:my-coll [1 2 3]})
Now you can request this value with the query [:my-coll] and it will return {:my-coll [1 2 3]}
#2021-11-0814:04Nikolas PafitisThanks alot#2021-11-0822:22wilkerlucioAlso its common that the items of collection are also maps, so they enable further processing using sub queries#2021-11-0817:59markaddlemanI have some query plans that take upward of 6 seconds to compute. I'm sure that I could simplify my resolvers to improve this but I don't have time to dig into it right now. In the short term, I'd like to use Redis for the plan cache. Of course, I don't want to use old plans from the cache, so I'm thinking that I should incorporate one of the Pathom indices into the cache key. I think the :com.wsscode.pathom3.connect.indexes/index-oir is a good index to use. Thoughts?#2021-11-0818:59wilkerlucioindex-oir is good, and to make it simpler to cache you can get the (hash index-oir) so you have a shorter thing to use as the cache key#2021-11-0819:10markaddlemanOut of curiosity: Is six seconds to calculate a plan concerning? I plan to investigate this more deeply but it's going to take a while before I get around to it#2021-11-0820:05wilkerlucioreally deps on how large is your query and how complex the attribute depth is#2021-11-0820:05wilkerlucioI think 6 seconds is really a lot, but if there is a lot of spreading, and a lot of OR nodes, than things can get nasty#2021-11-0820:06wilkerlucioif after inspection you see some way in which Pathom could do it better, we can work it out#2021-11-0820:06wilkerlucioin my experiments I get planning to finish mostly around 3 ~ 10 ms#2021-11-0820:46markaddlemanMy plan contains a lot of OR nodes. Especially early in the plan so that might compound the problem. It will be a few weeks before I can get into this in depth#2021-11-0821:03wilkerlucioOR paths are the trickiest thing to handle, I hope we find ways to improve it, but if you can reduce on your modeling, it surely helps#2021-11-1018:54markaddlemanfyi - I found some time to test the hypothesis that OR nodes cause the planning to take a long time. In short, yes. I with just a little hacking, I can get the planning down from 6000ms to about 20 milliseconds#2021-11-1018:58markaddlemanMy problem is that my app has about 25 resolvers that provide a public API that the client uses. These resolvers duplicate the output of internal resolvers so the plan that pathom produces is very large.#2021-11-1018:59markaddlemanBecause I know these resolvers are only used at the input edge of the plan, I can break query processing into two phases: first phase is a Pathom query that converts the public API to the internal API and the second phase is a Pathom query consistent of only the internal resolvers.#2021-11-1018:59markaddlemanMuch, much faster this way#2021-11-1018:59markaddlemanThanks for pointing me in the right direction!#2021-11-0822:03royalaid@wilkerlucio I think I found a small bug in the interaction of the mutation resolver plugin and the parallel parser#2021-11-0822:04wilkerluciogood catch, thanks!#2021-11-0822:03royalaidhttps://github.com/wilkerlucio/pathom3/pull/106#2021-11-0822:04royalaid#2021-11-0908:46Mark WardleHi all. Are there any issues to which I should be aware when using pathom3 with fulcro? I'm using pathom3 on my backend with a /api endpoint but have a re-frame front end - was thinking of making a switch as more sense as complexity grows managing all of the fetches and data freshness etc thank you. #2021-11-0910:10wilkerluciofor Fulcro it should make little to no difference, the interface between Pathom and Fulcro is EQL, which is unchanged between Pathom 2 and 3#2021-11-0910:11wilkerlucioone thing is that the fulcro tooling is behind in terms of Pathom Viz, which means in Fulcro Inspect you won't be able to see tracing of the runner graph from Pathom, but you can still use both tools at the same time (Fulcro inspect + Pathom Viz)#2021-11-1118:43Mark WardleThat's brilliant. Thank you Wilker! #2021-11-0918:12royalaid@wilkerlucio Hey we have been playing around with Pathom-Viz and have noticed that the index-explorer sometimes has problem reloading to the correct index and instead holds stale output. Unsure if its the way we hot reload or if the caching inside of Viz is doing it. I will try to get an isolated case later but wanted to call it to your attention#2021-11-0918:18wilkerluciothanks, please report as an issue on Pathom Viz when you find something, to be honest that would be lower priority for me, since a quick reload on the app can fix that, but if you or anyone wants to work on it I'll be glad to merge a fix#2021-11-1101:02mauricio.szaboHi, just checking if I can somehow make this modeling work. I have multiple resolvers for the same attributes, and they are prioritized and working. The problem is that some resolvers return more attributes than others. For example, resolver outputs :a, and resolver B outputs :a and :b, and have less priority.#2021-11-1101:04mauricio.szaboNow for the problem - I want resolver A to somehow signal "I found :a, and :b doesn't make sense in this context, so don't try to resolve this attribute" Currently I'm returning nil, but I really don't like to use nil in my code - is there a better way?#2021-11-1111:15jmayaalvhow do you determine that an attr doesn’t make sense in a given context?#2021-11-1112:35mauricio.szaboThat's what I want to know how to do it in Pathom šŸ˜„. But if you're asking like "more conceptually, in your code", it's because I'm trying to make a bunch of resolvers to drive Clojure plug-ins. In my specific case, I'm implementing "go to var definition" and friends. So, for example, if I want to "go to var defintion" for a var, and I find that the file is inside a JAR, it makes sense to get the contents of the file, uncompressed. But something things conflict: for example, if I find a file that is already on the current filesystem, and don't return its contents, the resolvers will keep trying to find a resolver that may get the contents, and sometimes this translates to "doing the wrong thing"#2021-11-1114:16jmayaalvgot the point, actually i think we have a similar problem šŸ™‚ something we were going to try was a to create a plugin where you register the optional outputs of a given resolver. something like:
(defresolver a-resolver []
  {::pco/optional [:b :c]}
....)
and the plugin basically would add the nulls if not present. i am not a big fan either of this so it would be great to hear a different solution šŸ™‚
#2021-11-1112:38souenzzoWhat is the difference between eql/focus-subquery and eql/mask-query ?! There is some (historical) reason to have these two?#2021-11-1118:12lilactowni'm interested in this too#2021-11-1121:05wilkerlucioI took a look at the code, they really seem the same, my guess is that at some point I wanted the mask and couldn't find the focus-subquery and ended up remaking it... but by looking at the sources you should prefer focus-subquery, the implementation is more complete, and supports unions#2021-11-1121:06wilkerlucioI wanna look closer to make sure this is the case, if so I'll deprecate mask-subquery#2021-11-1122:00lilactownawesome ty!#2021-11-1123:17souenzzoThey behave a bit differently in some edge cases
((juxt (partial apply eql/mask-query)
   (partial apply eql/focus-subquery))
 [[:a {:b [:c]}]
  [:a {:b []}]])
=> [[:a {:b [:c]}] [:a {:b []}]]
#2021-11-1114:13pithyless@wilkerlucio I'm studying your pathom-viz codebase and I think you've got a typo: https://github.com/wilkerlucio/pathom-viz/blob/de5fcb8f6ddc0acfbc834cf0c811d1de36fa39d2/src/core/com/wsscode/pathom/viz/helpers.cljs#L346 (-swap! [a f x y more] ,,,) should be (-swap! [a f x y & more] ,,,) - right?#2021-11-1114:15pithylessIf so, FulcroComponentProp and ReactAtomState have the same bug. Unless I misunderstood the code.#2021-11-1114:18wilkerlucioI believe this is correct though, its a protocol related thing, I remember going over this in the past, but would be nice to validate it, if you look at the ISwap protocol, you will see this:#2021-11-1114:18wilkerlucio
(defprotocol ISwap
  "Protocol for adding swapping functionality."
  (-swap! [o f] [o f a] [o f a b] [o f a b xs]
    "Swaps the value of o to be (apply f current-value-of-atom args)."))
#2021-11-1114:27pithylessYou're right, the protocol is correct and I confused swap! with -swap!. Sorry for the false alarm :)#2021-11-1115:38sheluchinI'm a little confused about how to use p/trace-plugin with Fulcro RAD. I've added p/trace-plugin as part of the extra-plugins vector here https://github.com/fulcrologic/fulcro-rad-demo/blob/a55b33ae82f372c26ee51b67c6c9b74a07a2cbeb/src/xtdb/com/example/components/parser.clj#L48. Then what? Where can I see the trace results?#2021-11-1115:53sheluchinAh, I see now, the response contains some new information:
{:start 1,
  :path [],
  :duration 14,
  :details [{:event "trace-done", :duration 0, :start 15}],
  :children
  [...
#2021-11-1116:57sheluchinAny tips on how to get a visualization working? I see trace->viz but haven't yet figured out how to get that to work from my trace output.#2021-11-1121:06wilkerluciothis is intended to use with Fulcro Inspect or Pathom Viz, these tools have code to render this trace data#2021-11-1121:09sheluchinAlright, so I use Inspect. Where in there can I find the tracer stuff? Sorry, I must be missing something really obvious..#2021-11-1121:47wilkerluciomaybe not so obvious, but can you try running a query from fulcro inspect query tab? ensure the checkbox for ā€œRequest Traceā€ is on#2021-11-1121:48wilkerlucioto get the trace you need to add :pathom/trace to the query, this is what the checkbox there does#2021-11-1121:49wilkerluciowhat people usually do is some server config to always add that in dev mode#2021-11-1121:49sheluchinI do not see "Request Trace". You mean in the EQL tab, yeah?#2021-11-1121:51sheluchinIt looks like I have latest installed too, 3.0.5.#2021-11-1121:54sheluchinAh, under Network at the very bottom. I see the profiler there now. But I still don't see Request Trace anywhere.#2021-11-1122:00sheluchinI notice the profiler doesn't show up at the bottom for all requests in the Network tab. For example, if I create a request through my UI, that doesn't have the profiler timeline at the bottom. If I then press Send to query and re-issue the request from the EQL tab, the new log entry in Network does have the profiler timeline at the bottom. I guess this might be why I missed it before. I still don't see the checkbox you speak of though.#2021-11-1201:11wilkerluciothis is the server configuration part I talked about, for tracing data to be available you need to add :com.wsscode.pathom/trace to the query, a simple way to do that is to wrap your parser in another fn like:
(defn traced-parser [tx]
  (parser (conj tx :com.wsscode.pathom/trace))
But you can make this query modification in any layer that makes sense in your system, could be also from the UI for example, modifying the network request from the Fulcro remote https://blog.wsscode.com/pathom/v2/pathom/2.2.0/core/trace.html
#2021-11-1217:46sheluchinThanks @wilkerlucio.#2021-11-1202:11mauricio.szaboHi, @wilkerlucio. Just to complement what we've been talking earlier in private: in my Duck-REPLed, I need some attributes available in all nodes/subnodes/etc all the times. The way I did this was by creating this code: https://gitlab.com/clj-editors/duck-repled/-/blob/master/src/duck_repled/editor_resolvers.cljc#L8-11. It's a resolver that have higher priority than everyone else, and it adds every information that somebody might need from env. Turns out, that's the worst idea ever that someone can do :rolling_on_the_floor_laughing:. It means that this node will be considered every time, for every query and it's slows everything down to a crawl. So I decided to, before each query, create a resolver and register it with pci/register. This works way better. Now, do you see a problem by doing this approach? Can it confuse plan caches? Is it a better way? WDYT?#2021-11-1214:30wilkerlucioit wont confuse the cache because the cache key takes the hash of the indexes in consideration, so different indexes will end up with different cache keys#2021-11-1202:48mauricio.szaboBTW, I can't use Pathom-VIZ on this project: I keep getting errors like:
/home/mauricio/projects/duck-repled/.shadow-cljs/builds/tests/dev/out/cljs-runtime/cognitect/transit.cljs:418
        (WithMeta. (-with-meta ^not-native x nil) m)
        ^
TypeError: x.cljs$core$IWithMeta$_with_meta$arity$2 is not a function
#2021-11-1214:31wilkerluciogotta see whats trying to add mat to some custom type you have that don't support it#2021-11-1214:31wilkerlucioa dirty fix would be to support meta on your type, but that's more like pushing the problem under the carpet#2021-11-1216:35mauricio.szaboThe problem is that I have no idea what's that object, to be honest...#2021-11-1216:56wilkerlucioyou said you have a reify thing on your data, could that be it?#2021-11-1300:38mauricio.szaboNo, I converted it to defrecord and the problem is still there :(#2021-11-1213:03lance.paineFirst, thanks for your amazing efforts here, this is really a super exciting project. I've been having great fun getting to grips with it. I'm using fulcro with a sparql backend for an exploratory POC project in my spare time. My data is therefore, inherently in a graph. I'm trying to figure out how that best maps to pathom and eql. I've got a couple of questions, this might look long, but I promise it's probably quite simple! :-) 1. what's the pattern for transitive, but normalized attributes? In my app, (I know, an odd choice for a learning exploration, but it's an already in RDF/Sparql open dataset! ) legal-entities (LEs) have addresses for two different reasons. legAddr (legal-address) and hqAddr(headQuarters). addresses have attributes like city, country, streetAddress. I have an query, and resolvers that's fetching the list of LEs, and some details for each LE. What I'm trying to wrap my head around, is if I want the query to ALSO grab the city of the legAddr for each LE, what does that look like? In the underlying data, legAddr and hqAddr are IRIs (so just ids), should the resolver be returning those uris as idents so the data remains normalized, and traversable? Something like {:legAddr [:address "address:addr1"]}? At the moment I can individually fetch the address details with a specific query.
example data looks something like
{
:legal-entities [{
    :id "le1" 
    :name "my company"
    :legAddr "address:addr1"
    :hqAddr "address:addr1"
  }
]
:addresses {
  "address:addr1" {
		:city "Auckland"
  }
}
So i want a query something like
;; for every le, get its name, and the city of the legal address
[{:legal-entities [:name {:legAddr [:city]}]}
[:legal-entities [
  {:id "le1" :name "my company" :city "Auckland"
]]
I've tried adding an alias resolver for :legAddr->:addressbut it doesn't seem to get triggered. I only see idents talked about in the docs in the context of mapping to graphql. What am I missing?
#2021-11-1215:28Bjƶrn Ebbinghaus
(defresolver resolve-legal-entities []
  {::pc/outputs [{:legal-entities [:le/id]}]}
  ... )

(defresolver resolve-legal-entity [_ {id :le/id}]
  {::pc/inputs [:le/id]
   ::pc/outputs [:le/name :address/id]}
  ...)

(defresolver resolve-address [_ {id :address/id}]
  {::pc/inputs [:address/id]
   ::pc/outputs [:address/city]}  
This allows querying for the city directly:
{:legal-entities
  [:le/name
   :address/city]}
If you want this data nested. (For separate Fulcro components, for example) you can still use placeholders.
{:legal-entities
  [:le/name
   {:>/address [:address/city]}]}
#2021-11-1215:28lance.paineoh, am I overthinking this? I just tried something more like
(parser {} [{:address/legal-entities
               [:legal-entity/id
                :legal-entity/legalName
                :legal-entity/legAddr
                :address/city]}])
And get the flattened data i need, right now. But it doesn't keep the structure I'd want if i want the legalAddr/city vs the hqAddr/city
#2021-11-1215:29lance.paine@U4VT24ZM3 thanks! at a glance, I think you just answered my question as I was stumbling into an answer. It's a huge help šŸ™‡#2021-11-1215:30lance.paineI knew I'd seen this :>/ magic somewhere, but couldn't see it again!#2021-11-1215:32Bjƶrn EbbinghausIn the case of two (or-more) addresses, you have to decide what you need šŸ™‚ You could just add them as joins.
(defresolver resolve-legal-entity [_ {id :le/id}]
  {::pc/inputs [:le/id]
   ::pc/outputs [:le/name 
                 {:le/legal-address [:address/id]}
                 {:le/hq-address [:address/id]}]}
  ...)

;; Query 
{:legal-entities
 [:le/name
  {:le/legal-address [:address/city]}
  {:le/hq-address [:address/city]}]}
#2021-11-1215:35Bjƶrn EbbinghausOf course, you can mix them depending on your use case. For example:
(defresolver resolve-legal-entity [_ {id :le/id}]
  {::pc/inputs [:le/id]
   ::pc/outputs [:le/name 
                 :address/id ;; legal-address is almost always used
                 {:le/hq-address [:address/id]}]} ;; hq-address is rarely used.
  ...)
#2021-11-1215:35lance.paineoh, i see!#2021-11-1215:35lance.paineand i think you're also answering somewhat my question about returning the ident for the address.#2021-11-1215:37lance.paineat the moment, I'm not declaring the resolver like
(defresolver resolve-legal-entity [_ {id :le/id}]
  {::pc/inputs [:le/id]
   ::pc/outputs [:le/name 
                 {:le/legal-address [:address/id]}
                
  ...)

but like 
(defresolver resolve-legal-entity [_ {id :le/id}]
  {::pc/inputs [:le/id]
   ::pc/outputs [:le/name 
                 :le/legal-address 
                 
  ...)
#2021-11-1215:37lance.paineand yet things are magically working. am i missing something in functionality, or performance?#2021-11-1215:38lance.paineI just read today something like "tell the parser as much as you can and it will use it to try and do a better job"#2021-11-1215:43Bjƶrn EbbinghausYes, you want to tell the parser as much as possible. (Not just the parser, but also for documentation.) That the join works without you telling the parser before will depend on the implementation of the parser.#2021-11-1213:05lance.paineMy second question: 2. What's the pattern for attributes that might exist on any id? In sparql land, any attribute can be attached to anything. So for instance, any IRI might have an rdfs:label attribute. Do I need to make a union query for all possible types in my system? Or alias every ident resolver to a label resolver? Or something else entirely?#2021-11-1215:52Bjƶrn EbbinghausIn this case, you can add a one-way alias:
(pc/alias-resolver :address/id :rdfs/id)

(defresolver resolve-label [_ {id :rdfs/id}]
  {::pc/input [:rdfs/id]
   ::pc/output [:rdfs/label]}
  ...)

;; Query:
{[:address/id 42]
 [:adress/city
  :rdfs/label]}
#2021-11-1216:00Bjƶrn EbbinghausThe question you have to ask yourself is whether this is a real use case. When you know which model supports which attribute, I wouldn't generalize this and just write:
(defresolver resolve-address [_ {id :address/id}]
  {::pc/input [:address/id]
   ::pc/output [:address/city :address/label]}
  ...)
#2021-11-1513:54lance.paineThanks @U4VT24ZM3, that's really helpful. Yes. it's possible I'm thinking about this from the perspective of the RDF/Sparql land. If you'll allow me, I'll ponder out loud here, to try and solidify my understanding. In the rdf land it's not uncommon to think of something deployable API like a 'label service', which given an entity ID (and a language) returns the appropriate label. This becomes less necessary with something like pathom in the mix. My end goal here is too look at how fulcro has done the likes of RAD backed by datomic, and make this datadriven from sparql. But I want to understand good manual practices, first. It seems the impedance mismatch here is that the coordinates for a datapoint in RDF are the entity (subject), and the predicate (attribute) -> object (in this example, the label), coloured by the open world philosophy (any one can say anything about anything). Whereas the pathom approach encodes an aspect of the type of the subject in the namespace of the attribute. This helps us by allowing the parser to hunt the graph, but has also closed the world, somewhat. By contrast the namespaces on predicates in rdf land are either some statement about who/what org has defined an ontology more so than what type it's expected to be applied on (for that, we've OWL and shacl).#2021-11-1516:17Bjƶrn Ebbinghaus> Whereas the pathom approach encodes an aspect of the type of the subject in the namespace of the attribute. This helps us by allowing the parser to hunt the graph, but has also closed the world, somewhat. Attribute names are up to you. It is not like they are "types". You can mix and match everything. You only have to define "edges" between your attributes. And just because I said you should define as much as possible, doesn't mean you have to. šŸ™‚ The Datomic RAD backend generates "static" resolvers based on a given schema. What may be interesting for you are "dynamic resolvers". Pathom 2 "technically" has support for them, but Pathom 3 will have "real" support. https://blog.wsscode.com/pathom-3-is-coming/#dynamic-resolvers-in-pathom-2 Maybe have a look at: https://github.com/wilkerlucio/pathom-datomic Which seems to be what you are trying to build for SPARQL.#2021-11-1516:24Bjƶrn EbbinghausBut considering dynamic resolvers, you should consult @wilkerlucio I know they exist and what they are for, but that's it. šŸ™‚#2021-11-1517:27lance.paineThanks, I'd seen dynamic resolvers were better supported in pathom3, so hopeful for that. No matter whether I generate static resolvers, properly support dynamic resolvers, or other, the expressive power and succinctness is pretty cool šŸ™‚#2021-11-1517:28lance.painei don't think I'd seen @wilkerlucio s pathom-datomic! šŸ™‚#2021-11-1216:47mauricio.szabo@wilkerlucio I'm having a hard time debugging an issue: I currently have the following resolver:
(connect/defresolver meta-for-clj-var
  [{:keys [var/fqn repl/clj repl/kind]}]
  {::pco/input [ :var/fqn  :repl/clj :repl/kind]
   ::pco/output [:var/meta]}

  (when (= :cljs kind)
    (eval-for-meta clj fqn 'user)))
It is never called in my graph. Even when I query for the inputs, they all resolve correctly, but the meta-for-clj-var itself never is called.
#2021-11-1216:48mauricio.szaboWhat I found is that if I add (pco/? :repl/clj) instead of asking for :repl/clj, it does resolve (and the input is present on the body). How do I debug this, to be able to even open an issue?#2021-11-1216:54wilkerlucioyou should try to reduce the case until you can find where exactly its failing to resolve, reporting with a simple and clear example pointing the issue is the best#2021-11-1218:59mauricio.szaboOk, it was 100% my fault, but I opened an issue even them because I think the behavior is strange: https://github.com/wilkerlucio/pathom3/issues/109 Long story short: the resolver for :repl/clj had a typo in the output. Even then, :repl/clj "leaked" through the graph, and in some places it was being accepted as a valid input#2021-11-1218:59mauricio.szaboOk, it was 100% my fault, but I opened an issue even them because I think the behavior is strange: https://github.com/wilkerlucio/pathom3/issues/109 Long story short: the resolver for :repl/clj had a typo in the output. Even then, :repl/clj "leaked" through the graph, and in some places it was being accepted as a valid input#2021-11-1303:13Pragyan TripathiI am trying to use Pathom Viz. I do my development within a vagrant box. I couldn't find anyway to connect Pathom Viz the pathom server running within the vagrant VM. Is there a way to connect it to remote pathom server. I see there a button to add a http connection. But I am not sure how do I find the connection information. Would appreciate any help in it.#2021-11-1407:22jmayaalvIf you have rest pathom endpoint you can just enter the url in Pathom Viz and you will have the same behavior. http://localhost:8080/pathom#2021-11-1407:23jmayaalvIn fact we normally use this as it is easier to see changes in the resolvers without reloading the index manually.#2021-11-1411:45Pragyan TripathiCan you point me to the doc where I can learn how to configure REST pathom endpoint? or any sample configuration?#2021-11-1506:34jmayaalvhttps://github.com/wilkerlucio/presentation-data-navigation-with-pathom3/blob/main/src/main/com/wsscode/presentations/pathom3_data_nav/pathom_server.clj#2021-11-1506:36jmayaalvif you want automatic reloads without need to refresh the registry manually you would need to tweak a bit the interceptor so that you reload the index on each request (do this only for dev mode)#2021-11-1406:46Pragyan Tripathi@wilkerlucio I was trying to experiment with pathom3-datomic and realized there's an issue with pick-ident-key method when using latest pathom3. I have created a PR to resolve the issue: #2021-11-1619:09wilkerluciothanks, I'll take a look on it later today šŸ‘#2021-11-1513:20dehliHi šŸ‘‹ From the pathom3 docs I see > Mutations are the first thing the runner executes, this way you know the reads from the query will have update values, in case the mutation affects something related to them. Is this statement still true with ::p.a.eql/parallel? true#2021-11-1619:08wilkerlucioyes, but with some caveats 1. if a mutation is async, other mutations may run at the same time 2. mutation sub-queries will run before the main query part (that's also true for other runner modes) other than that, the read part should only run after the mutations are done#2021-11-1620:10dehliPerfect! That’s what I was hoping for but wanted to confirm. Thanks!#2021-11-1513:24mauricio.szaboHi, I was just checking my code with strict and lenient mode, and I found something strange: in Lenient mode, sometimes an attribute will resolve (and in strict, it'll throw - always). Looking at the graph, it seems that the bug is on my side, but this opens a question: does lenient mode "tries hard" to fulfill the attribute? Or it's an implementation detail that may go away in the future?#2021-11-1619:09wilkerluciothis is more related to cases where output descriptions are missing, for those cases lenient may get a result where strict does not#2021-11-1803:14mauricio.szabo@U066U8JQJ what do you mean output descriptions missing? You mean when the resolver does not declare that it'll output a key, and it does output it anyway?#2021-11-1811:09wilkerlucioyes#2021-11-1813:09mauricio.szaboOk, that's not what's happening. What is happening is that I have multiple resolvers for :text/contents. For example, :text/current-var can return {:text/contents "foobar" :text/range [[0 1] [0 7]]}. There's also a resolver for :text/top-block. This resolver expects :text/contents and :text/range to get the "top block" of that content in that specific selection. So, suppose that I ask for {:text/current-var [:text/top-block]}. This should not resolve - :text/contents will always be a var, and :text/range will not be pointing to a block. The issue is that, on strict mode, this works as expected - it returns an error. But on lenient mode, it somehow resolves. I tried to debug it, and found that it indeed found an error in the top-block resolver, then it tried with different :text/contents that have lower priority. Is this expected?#2021-11-1814:02wilkerluciohard to understand from text, can you make a repro demonstrating it?#2021-11-1617:32tony.kayI see a final-value? predicate in operation…is this similar to using final in P2? If so, I don’t see a support function for setting final on a resolver return.#2021-11-1617:59wilkerlucioyes, same thing, I can add a support fn for it, can you open an issue for it?#2021-11-1617:59tony.kaysure#2021-11-1618:00tony.kayhttps://github.com/wilkerlucio/pathom3/issues/111#2021-11-1618:01wilkerluciothanks#2021-11-1618:33wilkerluciopco/final-value landed on main#2021-11-1618:33tony.kayk#2021-11-1618:34tony.kaydid you alraedy have a function like this (just wrote this), and does it look ok, and do you want it in your lib if you don’t already have it?
(defn- p2-resolver? [r] (and (map? r) (contains? r :com.wsscode.pathom.connect/resolve)))
(defn- p2-mutation? [r] (and (map? r) (contains? r :com.wsscode.pathom.connect/mutate)))
(defn- p2? [r] (or (p2-resolver? r) (p2-mutation? r)))

(defn pathom2->pathom3
  "Converts a Pathom 2 resolver or mutation into one that will work with Pathom 3.

  Pathom 2 uses plain maps for these, and the following keys are recognized and supported:

  ::pc/sym -> ::pco/op-name
  ::pc/input -> ::pco/input as EQL
  ::pc/output -> ::pco/output
  ::pc/transform -> applied before conversion
  ::pc/mutate
  ::pc/resolve

  Returns the input unchanged of the given item is not a p2 artifact.

  NOTE: Any `transform` is applied at conversion time. Also, if your Pathom 2 resolver returns a value
  using Pathom 2 `final`, then that will not be converted into Pathom 3 by this function.

  You should manually convert that resolver by hand and use the new final support in Pathom 3.
   "
  [resolver-or-mutation]
  (if (p2? resolver-or-mutation)
    (let [{:com.wsscode.pathom.connect/keys [transform]} resolver-or-mutation
          {:com.wsscode.pathom.connect/keys [resolve sym input output mutate]} (cond-> resolver-or-mutation
                                                                                 transform (transform))
          config (cond-> {}
                   input (assoc ::pco/input (vec input))
                   output (assoc ::pco/output output))]
      (if resolve
        (pco/resolver sym config resolve)
        (pco/mutation sym config mutate)))
    resolver-or-mutation))
#2021-11-1618:35wilkerlucioI dont have, I think this kind of code could live in a separate library, something like com.wsscode/pathom3-upgrade#2021-11-1618:35tony.kayit has no hard dep on p2, since it is just keywords#2021-11-1618:36tony.kaybut if you’re also going to make compat stuff for plugins it could make sense.#2021-11-1618:37wilkerlucioyeah, just don't wanna mix upgrade related code in the main repo#2021-11-1618:37tony.kayk#2021-11-1617:59zhuxun2Is there a way to retrieve the final ::p/env after the entire parsing?#2021-11-1618:51wilkerlucionothing built-in for that#2021-11-1618:51tony.kayIn Pathom 2 you can get parser from env to execute new EQL within a resolver…is this possible in 3?#2021-11-1618:52wilkerlucioyou don't need to, because the running process now is based on env, you can use it directly with p.eql/process#2021-11-1618:52tony.kayoh right…that makes sense#2021-11-1621:25souenzzoin pathom2 we can do a resolver that uses parser from env and works for both async and non-async process We can't do that in pathom3#2021-11-1621:35wilkerlucio@U2J4FRT2T what you mean? it should work on sync and async on Pathom 3 too, did you find something different?#2021-11-1621:36wilkerlucio(one difference is that you have to be aware which one it is on 3, so you can use the proper process fn, from p.eql or p.a.eql)#2021-11-1622:47wilkerlucioaltough, you can know that, because Pathom fills that in the env when its async it puts ::p.a.eql/async-runner? true into it#2021-11-2216:10mauricio.szaboWait, this does solve some of the problems I'm having! WDYT about adding it somehow in the documentation? That's something I never though it would be useful šŸ™‚#2021-11-1622:47wilkerlucio#2021-11-2315:09Jakub Holý (HolyJak)@wilkerlucio FYI https://edn-query-language.org/ gives > 404 There isn't a GitHub Pages site here.#2021-11-2315:35wilkerluciosomething strange is going on, the setup on my DNS seems correct but Github keeps complaining about it#2021-11-2315:36Jakub Holý (HolyJak)The DevOps joys. Good luck with it!#2021-11-2315:36Jakub Holý (HolyJak)I run https://holyjak.cz/ on GH as well, let me know if you want to compare my setup with yours#2021-11-2315:50wilkerlucioyeah, I mean, Pathom 3 and 2 are also there#2021-11-2315:50wilkerlucioI double checked, they are pretty much the same config, I think GH is trolling me šŸ˜›#2021-11-2315:52wilkerluciowell, its back : http://edn-query-language.org/eql/1.0.0/what-is-eql.html#2021-11-2315:52wilkerlucioI just kept saving it 🤷#2021-11-2315:52wilkerluciothanks for bringing it up#2021-11-2315:53Jakub Holý (HolyJak)thx for fixing so quickly!#2021-11-2315:22Jakub Holý (HolyJak)@wilkerlucio I think it would be helpful if both https://blog.wsscode.com/pathom/v2/pathom/2.2.0/connect/resolvers.html and https://pathom3.wsscode.com/docs/resolvers also mentioned what are the expected outputs of a resolver - i.e. that it always is a map representing a single data entity or wrapping a sequence of data entity maps (where I am unsure what "sequence" should be - can it be any of set/list/vector or does it need to be a vector? Regarding normalization of a seq. of entities in Fulcro it seems to onyl handle vectors of these) šŸ™#2021-11-2315:37wilkerlucioa sequence is a value as any other, I think its more accurate to think of a resolver response as a bag of labeled data#2021-11-2315:38wilkerluciobecause its not even the entity itself, since that's is a larger construct, from the user query#2021-11-2315:39wilkerlucioPathom 3 is made to handle any kind of sequences, but vectors still preferred, you can't batch if the value has a set or a list in the path for that entry (except on Parallel Runner, because it handles batching differently)#2021-11-2315:40Jakub Holý (HolyJak)I see, thank you. It was not clear to me from the current docs what sequences to use / prefer (though it may be just my text scanning skills that are lacking :))#2021-11-2315:41wilkerlucioIm down for improving on docs as always, if you have a specific suggestion I'll love to take PR for it#2021-11-2315:42wilkerluciothe getting started section has something that's more like what you suggested I think#2021-11-2315:55Jakub Holý (HolyJak)yes, that is what I have been looking for. I would add to 2. the mention of that vectors for sub-entities are preferred and perhaps replicate that info on the resolvers page.#2021-11-2315:55Jakub Holý (HolyJak)I would send a PR if I knew what exactly and where to put šŸ˜…#2021-11-2323:33wilkerlucio
#2021-11-2403:03markaddlemanAWESOME!#2021-11-2404:49Pragyan TripathiOh wow this so amazing.. How can we configure it. I am using both Pathom Viz, and portal would like to integrate them into one..#2021-11-2914:04souenzzonice to know that portal has cursive integration#2021-11-2411:46Jakub Holý (HolyJak)A question about P3 and error handling. According to https://pathom3.wsscode.com/docs/error-handling#optionality, this
(p.eql/process
    (pci/register
      (pco/resolver 'error
        {::pco/output [:error]}
        (fn [_ _]
          (throw (ex-info "Deu ruim." {})))))
    [(pco/? :error)]))
should throw an exception but returns {} using Pathom 2021.07.10-alpha Why? Changing the query to [:error ::pcr/attribute-errors] does not influence the result.
#2021-11-2412:54wilkerlucioI just tried to reproduce but I see an error when I run this code here :thinking_face:#2021-11-2412:54wilkerlucio#2021-11-2412:55wilkerluciobut try upgrading, latest is 2021.11.16-alpha#2021-11-2412:55wilkerlucio(I tested over the main branch here)#2021-11-2508:27Jakub Holý (HolyJak)I have changed to that version and now: 1. The query [(pco/? :error)] returns nil 2. The query [:error] throws an exception#2021-11-2422:09zeitsteinI noticed something while transitioning my resolvers from Pathom 2 to 3. Consider the following resolvers:
(pco/defresolver main-tree-resolver []
  {::pco/output [{:side [:root/id {:root/list [:id]}]}]}
  {:main {:root/id :main :root/list [{:id 1} {:id 2}]}})  ; {:id 1}

(pco/defresolver side-tree-resolver []
  {::pco/output [{:side [:root/id {:root/list [:id]}]}]}
  {:side {:root/id :side :root/list [[:id 1] [:id 2]]}})  ; [:id 1]

(pco/defresolver id-resolver [{id :id}]
  {::pco/output [:id :text]}
  (...)
Running the EQL query
[{:main [:root/id {:root/list [:id :text]}]}]
resolves everything, while the same query for :side doesn't resolve elements from :root/list. Which is fine (though, not how it worked for Pathom 2). But consider that main-tree-resolver doesn't return a :root/list vector of Fulcro-style idents, so it could not feed the client database. So, I guess I'm making a mistake somewhere?
#2021-11-2422:38wilkerluciothe example here has something wrong, the main-tree-resolver output has :side at root#2021-11-2422:38wilkerluciobut about [:id 1], to Pathom that's a vector with two elements, the concept of ident pointer like that is Fulcro only, in Pathom each item must always be a map with the available context (like at the main-tree-resolver values)#2021-11-2422:39wilkerlucioseems strange that this works on P2, the premise is the same there, maybe some plugins, or using some custom reader could change the behavior, but I dont expect that work by default#2021-11-2422:51zeitstein> the example here has something wrong, theĀ `main-tree-resolver`Ā output hasĀ `:side`Ā at root Copying error, sorry. This was the parser I used for P2 with Fulcro's mock-http-server:
(def pathom-parser
  (p/parallel-parser
   {::p/env     {::p/reader [p/map-reader
                             pc/parallel-reader
                             pc/open-ident-reader]}
    ::p/mutate  pc/mutate-async
    ::p/plugins [(pc/connect-plugin {::pc/register resolvers})
                 p/error-handler-plugin
                 p/request-cache-plugin
                 (p/post-process-parser-plugin p/elide-not-found)]}))
#2021-11-2422:57zeitsteinAnyway, thanks for clearing it up.#2021-11-2423:04zeitsteinBtw, Pathom 3 docs provide a very good introduction. Good job!#2021-11-2819:07Piotr RoterskiWhat is the idiomatic pathom3 equivalent of pathom2's :com.wsscode.pathom.connect/mutation-join-globals ?#2021-11-2819:07Piotr Roterskicontext: https://github.com/fulcrologic/fulcro-rad-demo/pull/34#2021-11-2920:21wilkerluciohello, humm, nothing yet, maybe it can be done with a mutation wrapper plugin#2022-11-3011:32Quentin Le GuennecHello, any update on this?#2022-11-3020:17wilkerlucioyes, eg: :com.wsscode.pathom3.format.eql/map-select-include #{:attribute :other} and will have the same effect as the mutation-join-globals, but keep in mind this will also affect every entity (not just mutation entites)#2021-11-2920:18Drew Verleecan some explain (re-explain) how the role of the "entity" or "entity tree" the 2nd optional arg passed to p.eql/process https://cljdoc.org/d/com.wsscode/pathom3/2021.08.14-alpha/api/com.wsscode.pathom3.interface.eql#process From here https://pathom3.wsscode.com/docs/environment/#entity It says it can provide data to the entity map, this would seem to be useful in testing and there is a NS for helpers on merging data in https://cljdoc.org/d/com.wsscode/pathom3/2021.07.19-alpha/api/com.wsscode.pathom3.entity-tree#create-entity But their not well documented (from my limited perspective). And i don't see any examples. I have been randomly throwing data in this entity-tree in hopes it would somehow show up in my tests. But a graph needs entity-attribute-value relationships and a "map" can't do that, which is where i assume the helpers come in?#2021-11-2920:23wilkerluciohello @U0DJ4T5U1, the entity is the initial data context, in the past Pathom made easy to provide a single attribute to start the processing (usually some id) via idents. the entity is the idea to extend that, and support an arbitrary bag of data to be available at hte process start, so pathom can fill the rest using this data as the bootstrap data#2021-11-2920:23wilkerluciomakes sense?#2021-11-2920:23wilkerlucioanother way to say it: the entity is the data you already know before you start resolving#2021-11-2920:24wilkerluciothe one in env is the cached entity so far, that value is an atom that starts with the entity (that you provide to process), but changes as attributes get resolved#2021-11-2923:53Drew VerleeIt does make sense, it might help to post a couple examples of working with it using the helper functions to get a sense of whats possible. As it stands, i just assoc keys and values directly into it.#2021-11-3014:07Mark WardleHi all. I have started using batch resolvers but I am getting WARN messages in my logs. I have an optional relationship from an encounter to the form_edss (although not marked as optional) that will return nil if there is no corresponding form of that type for the encounter. Tue Nov 30 12:53:38 UTC 2021 WARN :com.wsscode.pathom3.connect.runner/event-batch-unsupported - {:com.wsscode.pathom3.path/path [pc4.rsdb/search-patient-by-pseudonym :t_patient/encounters 3], :com.wsscode.pathom3.connect.operation/op-name com.eldrix.pc4.server.rsdb/encounter->form_edss} From https://github.com/wardle/pc4/blob/49cdaee6031997c3bf1290de25f9df51cacaa4ed/pc4-server/src/com/eldrix/pc4/server/rsdb.clj#L514 : (pco/defresolver encounter->form_edss [{:com.eldrix.rsdb/keys [conn]} encounters] {::pco/input [:t_encounter/id] ::pco/output [{:t_encounter/form_edss [:t_form_edss/id :t_form_edss/edss :t_form_edss/edss_score :t_form_edss_fs/id :t_form_edss_fs/edss_score]}] ::pco/batch? true} (map #(when-not (every? nil? (vals %)) (hash-map :t_encounter/form_edss %)) (forms/encounters->form_edss conn encounters))) Can I safely ignore these warnings, as the results see right to me on testing?#2021-11-3014:21wilkerluciohello, you shouldn't ignore those šŸ™‚ batch requires that the path to access the data is "randomly accessible", this means the path from the the root to the entity needs random access all the all, mas have that, vectors have that, but other types of lists and sets don't so you should check if all collections in the path for your batch are vectors ,otherwise pathom isn't actually batching, just running one by one#2021-11-3014:22Mark WardleGot it. Thanks Wilker!#2021-12-0303:33AbhinavI'm relatively new to pathom and EQL. I was wondering what the syntax for predicates is in EQL. for e.g. If I want the name of a product and its cost I can get it with this query. [:product-name :product-cost] but what if I want the name and cost of products that cost less than 100$ or something. how would I express that in EQL? Thanks#2021-12-0308:13dehliYou're looking for parameters. https://pathom3.wsscode.com/docs/resolvers/#parameters So that might look like
[{(:acme/products
   {:max-cost 100})
  [:acme/product-name
   :acme/product-cost]}]
Let us know if you have any follow up questions!
#2021-12-0310:32Abhinavomg yes. thank you so much#2021-12-0717:03Jakub Holý (HolyJak)Is it possible to make a query like [{:some/entity [*]}] i.e. one that would fetch all attributes of an entity? Fulcro supports '* but Pathom does not, right? I.e. I must list the attributes I want?#2021-12-0717:12zeitsteinhttps://pathom3.wsscode.com/docs/eql#wildcard like it should be possible?#2021-12-0717:36Jakub Holý (HolyJak)I forgot to mention I have P2, though an answer for P3 is also interesting#2021-12-0717:37wilkerlucioits important to understand what * means for Pathom, and that is "give me all the data you loaded so far", which is different from "all possible data for this entity"#2021-12-0717:37wilkerlucioso, for instance, a query like: [{[:ident/id 123] [*]}] will always return just {:ident/id 123}, because no resolver is triggered in this procress#2021-12-0717:38wilkerluciobut something like [{[:customer/id 123] [:customer/name *]}] might return a map with many attributes, if this system has some customer-by-id resolver that returns 8 customer properties, all those 8 would get out as a consequence of *#2021-12-0717:39wilkerluciofor a time there was a consideration to have another special one, like **, to get everything, but that never moved forward, and Im not convinced its a good idea at this moment, because the way Pathom works, its likely that in a lot of cases this might never finishes processing due to the amount of possibilities there#2021-12-0808:09Jakub Holý (HolyJak)Thank you! I am not trying to express a need for it, I was just curious. Thank you!#2021-12-2014:02tami5Interesting, it makes sense to support such a thing for something like a details page of a post.#2021-12-2014:38wilkerlucio@U02PHJ3C1QV that only works up to some point, for instance, a system will tend to expand, and even a blog post may have data you don't wanna fetch (like internal metrics), due to this is better to think about always expressing exactly what you need, and update the query in case you need to display more things, this will keep things predictable, otherwise you may accidentaly start fetching tons of unnecessary data#2021-12-2014:44tami5@U066U8JQJ oh that makes prefect sense. Thank you#2021-12-0821:24Braden Shepherdsonis pathom 3 sufficiently usable/documented for someone • writing a Fulcro RAD app • that isn't for production use • new-ish to Pathom ?#2021-12-0822:02wilkerlucioyes, RAD is going to isolate most of the things for you, and for the most things you will need to do with Pathom directly, the surface API barely changed from 2 to 3 (the part of defining resolvers)#2021-12-0822:03wilkerluciohttps://github.com/fulcrologic/fulcro-rad-demo/tree/pathom3#2021-12-0822:18Braden Shepherdsoncool, thanks!#2021-12-1414:28tony.kaySee the pathom3 branch of the RAD demo#2021-12-0905:16AbhinavHi, I'm relatively new to pathom, so I'm not sure how to frame this question but How do I pass the results of a join to a resolver? here's what I'm trying to do I have a query [:patients] that returns some data
{:patients
 ({:patient-id 2, :patient-name "John"}
  {:patient-id 3, :patient-name "Tina"}
  {:patient-id 0, :patient-name "Richard"}
  {:patient-id 1, :patient-name "Eve"})}
I have another query that does a join [{:patients [:patient-name :bill]}] to give me this
{:patients
 [{:patient-name "John", :bill 250}
  {:patient-name "Tina", :bill 90}
  {:patient-name "Richard", :bill 100}
  {:patient-name "Eve", :bill 120}]}
I want to take this result and sort this by :bill how can I achieve this? what will the query look like and how do I write a resolver for this? also, :patients` has a separate resolver and :bill has a separate resolver. (I am on pathom2) Thanks in advance
#2021-12-0911:44bgthe query (resolver) that’s returning the joined data should take a parameter that will specify the sort-key. sample eql could look like this
[{(:patients {:sort-by :bill}) 
  [:patient-name
   :bill]}]
#2021-12-0914:11Abhinavah, I don't have a resolver that returns the joined data. the open-ident-reader does that for me. I just have two resolvers, one for :patient and one for :bill I should mention the bill resolver uses the :patient-id to resolve the billing data for the patient ( I guess that's why the open-ident-reader resolves it for me)#2021-12-1009:32wilkerluciothere is no sorting built-in in Pathom, usually what I do is to sort the data after reading it (usually on the client side)#2021-12-1009:33wilkerlucioyou have the option to use params to implement sorting, but usually I recomend avoiding this, instead think if you really need a custom sorting, usually the lists tend to have a "natural order", which you should use in the resolver#2021-12-1414:28abdullahibrahi guys, anybody can help in this error?#2021-12-1414:28abdullahibrathat occurred when the frontend waits the backend to give back a results and it exceeds the timtout#2021-12-1414:28abdullahibrais there a way to increase this timeout for specific transactions#2021-12-1414:46wilkerlucioyes, you can increase it by setting :com.wsscode.pathom.parser/key-process-timeout 60000 (this is the default) in your env#2021-12-1414:46wilkerluciobut if its taking that long, is it expected?#2021-12-1414:47wilkerluciousually when it gets to this point is a sign that something isn't going well#2021-12-1414:48abdullahibraYes. I got it, Thank you šŸ™‚#2021-12-1820:22lgesslerhey #xtdb users: any thoughts on how to make xtdb's history features and pathom resolvers play nice? coming up with a few ideas... I think the most promising one goes like this: • add ring middleware to look for a query param like ?xtdb-validtime=... (or a cookie? not sure what the best http option here might be) • have the middleware extend the request (and therefore the pathom environment, in most pathom+ring setups) with a {:historical-db (xt/db node xtdb-validtime)} • modify resolvers to prefer :historical-db to :node when it is set would appreciate any thoughts on this approach#2021-12-1822:17lilactownI haven't used xtdb, but I imagine if it's similar to datomic having an :as-of parameter at the root of a query might be a neat way to handle that#2021-12-1822:17lilactownostensibly all nodes in a query should use the same db-time, right?#2021-12-1822:41lgesslerright, all entities for the query would be using the same db-time. that's an interesting idea too!#2021-12-1900:29wilkerlucioyes, thats a good way to handle, I suggest setting the :as-of in the env so its globally available#2021-12-2108:04bedersHi there, I’m using the parallel-parser on pathom 2.4.0. It seems that if one of my resolvers throws an exception it messes with the state-machine of the parallel-parser and from that point on - every 60 seconds (key-process-timeout), I’m getting a clojure.lang.ExceptionInfo: Parallel read timeout Anyone else experiencing this?#2021-12-2108:05bedersHere’s the relevant code:
(def eql-parser
  "Pathom EQL parser for Accounts.
  Keep public to allow access in dev mode, see user.clj."
  (p/parallel-parser
   {::p/env                               {::p/reader               [p/map-reader
                                                                     pc/parallel-reader
                                                                     pc/open-ident-reader
                                                                     p/env-placeholder-reader]
                                           ::p/placeholder-prefixes #{">"}}
    ::p/mutate                            pc/mutate-async
    ::p/plugins                           [(pc/connect-plugin {::pc/register registry})
                                           (p/env-plugin {::p/process-error (fn [env ex]
                                                                              (cast/with-context (select-keys env [:correlation-id])
                                                                                                 (cast/alert {:msg "PathomError Accounts"
                                                                                                              :ex  ex}))
                                                                              {::p/reader-error (.getMessage ex)})})
                                           p/error-handler-plugin]
    ::parser/external-wait-ignore-timeout 10000              ;; the default 3000 is a bit problematic
    }))

(defn eql
  [env eql-query]
  (async/<!! (eql-parser env eql-query)))
#2021-12-2113:51wilkerluciohello, can you send a full repro? I don't see the resolvers in this example#2021-12-2117:17bedersThanks for getting in touch Wilker. I’ll try to come up with a complete example. Do you see any issue with the error handler returning (::p/reader-error "…"} ?#2021-12-2119:04wilkerluciono worries, and I dont see any immediate issue on that#2021-12-2118:42lgesslersuppose my query is this and I have exactly one resolver for each attr:
[{:top-attr 
  [:A 
   {:B [:C]}]}]
if I change ::p/env in my resolver for :B, I should expect that these changes will be available for :C but that they may or may not be available for :A, right? (Since children have to be processed after parents, but sibling attrs may be processed in either order)
#2021-12-2119:06wilkerlucioyeah, changing the resolvers in mid-process sounds a dangerous game to play#2021-12-2119:06wilkerluciowhat are you trying to achive?#2021-12-2120:31lgessleryeah šŸ˜… well, long story short, I'm trying to resolve attrs (sometimes 3-4 joins deep) that need to be aware of a top-level ident join#2021-12-2120:32lgesslerProbably it's better to just write resolvers that express the whole path in ::pc/output and ::pc/input from the top-level ident to the attr instead of fiddling with the env, which is dangerous as you suggest#2021-12-2120:34lgessler(tangentially, now I'm wondering if I understand something right--`::p/env` additions from a resolver do not persist beyond the lifetime of the mutation/query that it was made in, right? global pathom env != tx env)#2021-12-2121:08wilkerluciocorrect, you can only affect env for children nodes, the initial motivation for it was to implement a sharding system, where some entities may change the shard they are (and that affects the children, because are related, and likely on the same shard)#2021-12-2121:08wilkerlucioyou can also consider Union queries, which is made to support branching on the query (so use query A if entity is like X...), but for many levels deeper it may not be so easy#2021-12-2217:49lgesslerhm ok yes, I'll consider whether unions might be a good fit. Thanks for the help as always šŸ™‚#2021-12-2217:37sheluchinWhat's the best learning path for Pathom right now? Straight to the Pathom3 docs and blog posts? Would I be missing anything by skipping Pathom2 docs?#2021-12-2217:38markaddlemanfwiw, I started with Pathom3 and the docs. I had no experience with Pathom2. It was pretty easy to learn.#2021-12-2218:57wilkerlucioits fine to start straight from 3, nothing to miss from pathom 2 docs šŸ™‚#2021-12-2219:20sheluchinThanks!#2021-12-2510:18jherrlinHappy holiday! Really happy that I finally got time to test out Pathom! I've made a couple of resolvers and got the basics to work. Now I wonder how to reason about resolvers and actions (side effecting stuff). I would like to unit test my resolvers and have come up with this. I assoc a function to each resolver map and that function will be used inside the resolver to return data. In production this functions may do a database or http request. But in the test suite it returns static (or generated) data. Is this a good way to do this? Are there dangerous things I missed?#2022-01-0119:10Drew Verleeyou can call your resolver like regular functions, so the same concepts apply for testing.#2022-01-0119:16Drew VerleeThat means you have consider the trade off in calling the "side effect" the reason people tend to mock it is because there is to high a cost. But thats a guideline not a rule.#2021-12-2510:18jherrlin#2022-01-0914:05wilkerlucioMinor release of Pathom 3 2022.01.09-alpha is out. This release fixes warnings in the CLJS side.#2022-01-2113:59wilkerlucio#2022-01-2618:03shvetsm@wilkerlucio Hey thank you so much for building this great framework. I have a couple questions 1. Say I have records with
[ {:value "A" :record.created-by-id "123" :record.last-modified-by-id "456" }
  {:value "B" :record.created-by-id "789" :record.last-modified-by-id "456" }]
and then I have a service that does a batch lookup for users. What is the recommended way of setting this lookup up if I would like to return something that assigns a name to both created-by and last-modified-by I am sure this is a common problem šŸ™‚. We have a lot of leeway with the shape of the returned data, though we do need a batch lookup. One will be ideal, we can def live with two (one for created one for last-modified). We do need to be able to tell which one is which. 2. We are starting development now. How alpha is Pathom3? I saw someone asking about parallel parser and such. We are not shy about using alpha (got a lot of mitosin floating on the project), but would thought I would ask. Once again thanks for all your hard work.
#2022-01-2618:18wilkerluciohello šŸ™‚ 1. not what you want to do, what you mean by assign a name to created-by and last-modified-by? how these data is related to the records you presented? 2. Pathom 3 has a mostly stable API already, not expecting that to have big changes, that said, the basic features (local processing and serial process) seem to be stable, I've been using in a couple of projects and there are other users here too. I would say that features like distributed graphs is the one to not have a lot of confidence at this point, not very used and likely to be bugged at this stage#2022-01-2618:23shvetsmok to be specific: I am modelling a resolver that gives me
[{:record/id "42" :record/created-by-id "user-id-1" :record/last-modified-by-id "user-id-2" ]
and so on I also have an api that can look up usernames in bulk so I can pass in
["user-id-1" "user-id-2"}]
and get back
[{:user/id "user-id-1 :user/display-name "Billy Bob"}
 {:user/id "user-id-2 :user/display-name "Jane Smith"}]
#2022-01-2618:28shvetsmSo what I am trying to do here is return something like
{:records [ {:record/id  "42" 
             :record/created-by-name "Billy Bob"
             :record/last-modified-by "Jane Smith"}]
#2022-01-2618:28shvetsmonce again it does not need to be this shape, just as long as I can navigate to the name#2022-01-2618:29shvetsmfor each created-by and last-modifed-by#2022-01-2618:29shvetsmthese values appear on 90% of our data, which needs to be displayed with the name on the UI#2022-01-2618:29wilkerluciocool, the trick is to change your bulk input, and label the data you already have, so instead of ["user-id-1" "user-id-2"], it would be more like: [{:user/id "user-id-1"} {:user/id "user-id-2"}]#2022-01-2618:29shvetsmI have that#2022-01-2618:30shvetsmI have the batch resolver working#2022-01-2618:31wilkerluciothat sounds correct to me, whats the part I can help you with?#2022-01-2618:31shvetsmI did an alias between :record/created-by-id and :user/id#2022-01-2618:31shvetsmbut I have 2 "user/ids" on each of this record#2022-01-2618:31shvetsmone for created by and one for last modified by#2022-01-2618:33shvetsm, how do I get both names to return for each record#2022-01-2618:33shvetsmI set up 2 aliases to map from created-by-id and last-modified-by-id for user/id#2022-01-2618:34shvetsmbut when I write my query I can only get 1 of the mappings#2022-01-2618:34shvetsmhere is my actual query
[{:notes [:note/id :note/created-by-id 
          :user/display-name]}]
 
#2022-01-2618:35shvetsmthere are two issues here. I don't know which one the user/display-name is (created by or last modified by)#2022-01-2618:36shvetsmand only 1 is selected#2022-01-2618:36shvetsmI tried to model it with a join [{:note/created-by-id [:user/id]}]#2022-01-2618:36shvetsmbut that does not work in batch#2022-01-2618:37shvetsmbecause the query results are nested now#2022-01-2618:37shvetsmI am asking because I think that this is an easy problem and I am just a noob šŸ™‚#2022-01-2618:41shvetsm
(pc/defresolver note-test-resolver [_ _]
  {::pc/output [{:notes [:user/id]}]}
  {:notes [{:note/id               "123"
            :note/created-by-id    "0495e640-b11c-4b58-a29d-8ce0ba485144"
            :note/last-modified-by "04bfa00e-d161-46b4-b2cf-d829905c86db"}
           {:note/id               "789"
            :note/created-by-id    "047ab33e-fef2-4c3d-9d36-f8d3b0170525"
            :note/last-modified-by "047893f7-e697-436f-8259-2b6e0ae45c1d"}
           {:note/id               "456"
            :note/created-by-id    "170aa5c2-bcab-4cc3-990e-2e67f996fc58"
            :note/last-modified-by "d85c0a12-22ff-40a2-a050-811606ee375e"}]})
#2022-01-2618:42shvetsm
(pc/defresolver user-resolver [_ -inputs]
  {::pc/input     #{:user/id}
   ::pc/output    [:user/display-name]
   ::pc/transform pc/transform-batch-resolver}
  (let [inputs (if (sequential? -inputs) -inputs [-inputs])
        users  (get users from db)
        items  (map (fn [user] {:user/id           (:id user)
                                :user/display-name (:display-name user)
                                :user/email        (:email user)}) users)]
    (pc/batch-restore-sort {::pc/inputs     inputs
                            ::pc/key        :user/id
                            ::batch-default (fn [] {:user/display-name "NO-NAME"})} items)))
#2022-01-2618:43shvetsm
(def registry
  [ note-test-resolver user-resolver
   (pc/alias-resolver :note/created-by-id :user/id)
   (pc/alias-resolver :note/last-modified-by-id :user/id)
   pc/index-explorer-resolver])
#2022-01-2618:43shvetsmthis is how I set it up#2022-01-2618:43shvetsmif it were just created-by I needed to lookup there would be 0 questions#2022-01-2621:45wilkerluciono worries, taking a look at your text now#2022-01-2621:46wilkerlucioif you have multiple options for one attribute, there will be ambiguity (which is nice to avoid when you can), having a nested key for each (one for created, one for last modified) can be a way to solve the ambiguity#2022-01-2621:48wilkerlucioso in this case, you can have two extra resolvers, as:
(pco/defresolver nav-user-created [{:keys [note/created-by-id]}]
  {::pco/output
   [{:note/created-by [:note/user-id]}]}
  {:note/created-by {:user/id created-by-id}})

(pco/defresolver nav-user-last-update [{:keys [note/last-modified-by-id]}]
  {::pco/output
   [{:note/last-modified-by [:note/user-id]}]}
  {:note/last-modified-by {:user/id last-modified-by-id}})
#2022-01-2621:49wilkerluciothen you can query:
[{:notes [:note/id {:note/created-by [:user/name]}]
`
#2022-01-2621:49wilkerlucioplease let me know how it goes#2022-01-2619:48dehliIf I have an attribute marked as optional in my request, is it expected that I’d get the Insufficient data calling resolver error message when trying to reach that attribute?#2022-01-2619:49markaddlemanDo you have two resolvers that output the same attribute?#2022-01-2619:49dehliI do#2022-01-2619:59markaddlemanI think this is a planning bug. I've been meaning to create a repro case for it#2022-01-2620:01dehligreat! ill try to create one as well. ill message here w/ the ticket when i do#2022-01-2620:01markaddlemanI think I got around it by avoiding optional entirely. I have one resolver that requires the attribute and one that does not. Then, I use priority to get the behavior that I want#2022-01-2621:44wilkerluciothe only way you may get that still is if some other required attribute depends on that, in this case it still invalid, otherwise please open an issue with a repro šŸ™‚#2022-01-2722:11markaddlemanI just filed https://github.com/wilkerlucio/pathom3/issues/120 with a repro case. I'm pretty sure it's a bug but I'm not sure if it's a planning thing or an overzealous checker#2022-01-2803:31wilkerluciothanks for that! fixed on main#2022-01-2814:42markaddlemanThanks for the quick turnaround!#2022-01-2803:41wilkerlucio#2022-01-2818:22cjsauerHello šŸ‘‹ I'm following the pathom3 graphql tutorial, and while it works fine for the Star Wars API, when I point it to the https://docs.gitlab.com/ee/api/graphql/getting_started.html I encounter consistent browser crashes (I'm working with cljs in the browser). Looking into the code, it seems the strategy is to load the entire schema into memory in order to produce the pathom indexes, but I think the sheer size of the Gitlab schema is causing problems in a browser environment (it seems roughly 25x larger than the Star Wars example). I realize this is a very experimental feature, but I'm just curious whether this is a known bottleneck?#2022-01-2818:51wilkerlucioI had the same issue, the problem is that Gitlab API is really huge, I have plans to get back on that, the issue is probably around some inefficiencies on handling nested inputs (which the Pathom GraphQL schema loader implementation uses a lot)#2022-01-2818:52wilkerluciothe temporary (not ideal) solution is to just wrap the REST API, the parts you use (that's what I'm doing until this gets figured)#2022-01-2819:36cjsauerAh okay, good to know. I'll look into the REST API. Thanks!#2022-02-0115:04wilkerlucio@cjsauer can you try again using latest Pathom and Pathom3 GraphQL? I just merged a significant improvement on Pathom 3 GraphQL regarding the schema process part of it, more details at: https://github.com/wilkerlucio/pathom3-graphql/pull/12#2022-02-0115:15cjsauerNice! I'll give it a try and let you know. #2022-02-0116:02cjsauer@wilkerlucio the schema loading of the gitlab API is now near instant on the latest commit! However, there may be a bug somewhere in the cljs side of things, but it could also very well be me doing something wrong. I keep running into the error below when attempting to run queries using the p.eql/process function. Here's the query I'm trying to run:
(p/let [result (p.eql/process env
                   [{:swapi.Root/allPeople
                     [{:swapi.PeopleConnection/people
                       [:swapi.Person/name
                        {:swapi.Person/filmConnection
                         [{:swapi.PersonFilmsConnection/films
                           [:swapi.Film/title]}]}]}]}])]
    (tap> result))
It's the same query from the https://pathom3.wsscode.com/docs/tutorials/graphql-integration/. And here's the error:
#2022-02-0116:40wilkerlucio@cjsauer can you try main again please? pushed some changes#2022-02-0116:44cjsauer@wilkerlucio that fixed it! It's working now for both the tutorial API and gitlab. Thank you for the quick fixes.#2022-02-0116:45wilkerluciothanks from bringing it up, its a push for making things happen šŸ™‚#2022-02-0116:52cjsauerOf course. Just noticed that the copy/paste brought in the wrong function name: https://github.com/wilkerlucio/pathom3-graphql/pull/13#2022-02-0115:04wilkerlucio@cjsauer can you try again using latest Pathom and Pathom3 GraphQL? I just merged a significant improvement on Pathom 3 GraphQL regarding the schema process part of it, more details at: https://github.com/wilkerlucio/pathom3-graphql/pull/12#2022-02-0116:02cjsauer@wilkerlucio the schema loading of the gitlab API is now near instant on the latest commit! However, there may be a bug somewhere in the cljs side of things, but it could also very well be me doing something wrong. I keep running into the error below when attempting to run queries using the p.eql/process function. Here's the query I'm trying to run:
(p/let [result (p.eql/process env
                   [{:swapi.Root/allPeople
                     [{:swapi.PeopleConnection/people
                       [:swapi.Person/name
                        {:swapi.Person/filmConnection
                         [{:swapi.PersonFilmsConnection/films
                           [:swapi.Film/title]}]}]}]}])]
    (tap> result))
It's the same query from the https://pathom3.wsscode.com/docs/tutorials/graphql-integration/. And here's the error:
#2022-02-0116:06cjsauerHere is my full cljs code, adapted from the tutorial page.#2022-02-0116:16cjsauerWhen I (tap> env) I can see that the indexes are indeed loaded, so it doesn't appear to be a schema loading issue. The gitlab API also shows up fine when I (tap> env).#2022-02-0116:22cjsauerHere is a test of the request-gitlab-graphql function:
(-> (request-gitlab-graphql "query { allPeople { people { name } } }")
      (p/then #(js/console.log %)))
Which indeed logs out the expected Star Wars character names. So it doesn't appear to be an issue there.
#2022-02-0200:09wilkerlucioReleased Pathom 3 2022.02.01-alpha 2022.02.01-1-alpha, this release contains some important fixes on optionals, thanks to @dehli for those reports • Fix error trigger when a dependency of some optional attribute fails (bug #126) • Fix planning errors on optional nested inputs (bug #127)#2022-02-0200:11dehliThanks for fixing them so quickly!#2022-02-0200:14wilkerluciofix -1 to remove a debug tap> that got in the release#2022-02-0213:32jgood@wilkerlucio I'm running into a weird issue. I'm using the async interface p.a.eql. When any resolver has a go block and the request has a palceholder: The resolver fails with :pcr/attribute-errors Error: Required attributes missing if the dependencies don't rely on some placeholder provided data -- even if the resolver has no dependencies.#2022-02-0214:07wilkerluciohello, thanks, can you open an issue with a repro example please?#2022-02-0223:48jgoodhttps://github.com/wilkerlucio/pathom3/issues/128#2022-02-0300:03wilkerlucioah, sorry, I just realized you are trying to use core.async there, have you imported the promesa bridges library? https://github.com/wilkerlucio/promesa-bridges#2022-02-0300:04wilkerlucioPathom 3 uses Promesa (isntead of Core.async) as the async manager tool, the promesa-bridges can make core.async compatible with promesa#2022-02-0300:04wilkerlucioplease let me know if that's the case, or if its something else#2022-02-0412:46jgoodI didn't include the imports in the example but I do use [com.wsscode.promesa.bridges.core-async]#2022-02-0412:47jgoodThe async resolver works as long as the resolver has an input that depends on something in the placeholder.#2022-02-0413:08jgoodI'm feeling more confident this is a bug in pathom and I've updated the issue example to include imports and more queries to show the problem.#2022-02-0413:39wilkerluciothanks, the example looks great, I'll have closer look later today#2022-02-0714:34wilkerluciosorry the delay, a quick update I can confirm the issue when using go blocks in the scenario that you described#2022-02-0714:35wilkerlucioI tested doing the same using promesa futures, and in this case it seems to work fine, I'll have to look further to understand what's causing the distinction#2022-02-0722:54wilkerluciopost a detailed response in the issue https://github.com/wilkerlucio/pathom3/issues/128#2022-02-0214:03sheluchinI'm running into an issue when executing this query through the latest viz: GraphQL version (works in the GH Explorer):
{
  viewer {
    repositories(first: 1) {
      edges {
        node {
          name
          issues {
            totalCount
          }
        }
      }
    }
  }
}
[{:github.Query/viewer
  [{(:github.User/repositories {:first 1})
    [{:github.RepositoryConnection/edges
      [{:github.RepositoryEdge/node
        [:github.Repository/name
         {:github.Repository/issues
          [:github.IssueConnection/totalCount]}]}]}]}]}]
#2022-02-0214:04sheluchin#2022-02-0214:14wilkerluciothis seems to be something weird with the Github implementation, Pathom GraphQL integration leverages the types to allow some polymorphic queries with ease, but seems like Github is not happy with that... here is the query that Pathom generates for this case:
query {
  __typename
  ... on Query {
    viewer {
      __typename
      ... on User {
        repositories(first: 10) {
          __typename
          ... on RepositoryConnection {
            edges {
              __typename
              ... on RepositoryEdge {
                node {
                  __typename
                  ... on Repository {
                    name
                    issues {
                      __typename
                      ... on IssueConnection {
                        totalCount
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}
#2022-02-0214:14wilkerlucioI wanna investigate closer where the real problem is, up to some level it works, but after it seems to break (just asking the name from viewer works fine for example)#2022-02-0214:16sheluchinYes, it breaks when walking into the :gh.Repo/issues edge and asking for the totalCount. Up to that point I didn't have any issues so far.#2022-02-0214:20wilkerlucioI think a way to improve that on Pathom side is to avoid using ... on Type when the attribute is targeting the same type as the current context#2022-02-0214:22wilkerlucio(I tried removing the ... on IssueConnection part and it did the trick to fix it)#2022-02-0217:38wilkerluciohttps://github.com/wilkerlucio/pathom3-graphql/issues/14#2022-02-0217:41sheluchinThanks @wilkerlucio. Glad to have alerted you of the issue. I'm looking forward to making use of the GraphQL stuff in Pathom šŸ™‚#2022-02-0217:44wilkerlucioI'll try to get to that later today, I have already an idea on how to fix it šŸ™‚#2022-02-0222:20wilkerlucio@alex.sheluchin fixed on main!#2022-02-0222:21sheluchin@wilkerlucio thank you, sir šŸ™ I will experiment some more with the GH API and let you know if there are other surprises.#2022-02-0300:02wilkerlucio@alex.sheluchin since you playing with Github, I just made a fix to root entries (old ident-map), and updated the example on Github to allow a query like this to work:
(p.eql/process env
  {:github.User/login "wilkerlucio" :github.Repository/name "pathom3"}
  [:github.Repository/homepageUrl])
Updated example https://github.com/wilkerlucio/pathom3-graphql/blob/main/demos/com/wsscode/pathom3/graphql/demos/github.clj
#2022-02-0419:53sheluchinHey @wilkerlucio, I'm trying to make the root entries example you showed work, but not quite getting it. I'm presuming in Pathom3 it's basically the same as Pathom2? ie the wrapper function to build the ident as recommended in the Pathom2's https://blog.wsscode.com/pathom/v2/pathom/2.2.0/graphql/fulcro.html#MultiInputIdents still applies?#2022-02-0419:59wilkerlucio@alex.sheluchin yes, but a bit different, in Pathom 3 we can now send root data much easier than we could in Pathom 2, the think about root entries mapping is that you can access GraphQL entry points without using the entry points, so you can for example provide a user login and ask for user data (instead of nesting over user(login: ...))#2022-02-0419:59wilkerluciothe code from the linked example, plus the
(p.eql/process env
  {:github.User/login "wilkerlucio" :github.Repository/name "pathom3"}
  [:github.Repository/homepageUrl])
should work, have you seem something bad while trying it?
#2022-02-0420:03wilkerluciobut about wrapping funcitons, no need to make them nowadays#2022-02-0420:05wilkerluciothat syntax doesn't even work anymore, idents can only send a single value, you can now use placeholders to send multiple data points, but there are also other options as: 1. second argument in p.eql/process to send data to the root 2. if using boundary interface, you can send the request as: {:pathom/entity {:entity/data "foo"} :pathom/tx [:query]}#2022-02-0420:59sheluchin@wilkerlucio I think I have something wrong with my setup right now. All queries just give me:
#object[java.util.concurrent.CompletableFuture 0x35e53559 "rejected"]
When I inspect env:
#object[clojure.lang.Var$Unbound 0x64a74b92 "Unbound: #'com.wsscode.pathom3.graphql.demos.github/env"]
And when trying to re-assign it with make-env, I get:
Execution error (AssertionError) at com.wsscode.pathom3.connect.indexes/register-mutation (indexes.cljc:194).
Assert failed: Tried to register duplicated mutation: com.wsscode.pathom.viz.ws-connector.pathom3/request-snapshots
(nil? (com.wsscode.pathom3.connect.indexes/mutation indexes op-name))
And it seems to stay that way after I shut everything down and retry. Do you know how I can get around it? Also, queries do work from Viz. Is it possible to use your snippet there? It expects a single vector/EQL query..
#2022-02-0421:04wilkerluciofrom viz you can use the new panel below the query to provide initial data#2022-02-0421:04wilkerluciocan you give a full example of the thing you see? are you using JVM or Browser pathom ?#2022-02-0421:12sheluchinI am on JVM, just following the example in com.wsscode.pathom3.graphql.demos.github.#2022-02-0421:14sheluchinThe modification I've made is here:
(defn make-env []
  (-> {}
      (p.gql/connect-graphql
        {::p.gql/namespace        "github"
         ::p.gql/root-entries-map {"repository" {"name"  ["Repository" "name"]
                                                 "owner" ["User" "login"]}
                                   "user"       {"login" ["User" "login"]}}}
        request)
      (p.connector/connect-env {::pvc/parser-id ::env
                                ::p.connector/async? true})
      ((requiring-resolve 'com.wsscode.pathom.viz.ws-connector.pathom3/connect-env)
       "gql-github")))
I've restarted everything, reconnected to my repl, and go to eval (def env (make-env)). This then gives me:
Execution error (AssertionError) at com.wsscode.pathom3.connect.indexes/register-mutation (indexes.cljc:194).
Assert failed: Tried to register duplicated mutation: com.wsscode.pathom.viz.ws-connector.pathom3/request-snapshots
(nil? (com.wsscode.pathom3.connect.indexes/mutation indexes op-name))
That's the full thing.
#2022-02-0421:20sheluchinRemoving the call to p.connector/connect-env from make-env does allow me to use that root entry query from Viz.#2022-02-0421:21sheluchinNo luck in my REPL though. I must be missing something basic about how to use it.#2022-02-0421:27sheluchinSorry @wilkerlucio, gotta be something on my end. I've set up reveal for tap> and now I can see the responses properly, and indeed the root entry query works! As for Viz, I didn't realize that bottom Entity Data window was for that purpose. It makes sense now. Appreciate your explanation. I will test a bunch of other GH queries and will let you know if anything doesn't work - hopefully stuff that isn't just related to my setup šŸ™‚#2022-02-0421:49wilkerlucioI see you are setting up the connector with ::p.connector/async? true, unless you using the async runner, that will make the thing fail#2022-02-0421:51wilkerlucioalso, the last line, the requiring-resolve is also adding the connector, so that's adding the connector twice, which will give the error of duplicated things you see#2022-02-0421:51wilkerluciousing the require-resolve is a way that I do it so I don't have to :require the viz namespace (makes easy to put in/out)#2022-02-0214:35imreHey @wilkerlucio, is there any kind of pathom2->3 migration guide available somewhere?#2022-02-0217:39wilkerlucionothing available at this time#2022-02-0217:40wilkerluciobut a summary: in general the resolvers should work fine, attention need to be taken if use stuff from env on the resolvers, also plugins would rewrite some rewriting because the entry points are different now#2022-02-0217:41imreon resolvers: if I read it correctly, input specification changed a bit as well, right?#2022-02-0217:42wilkerlucioyes, minor change, they changed from sets to vectors (EQL to be more precise, now that they accept nested inputs)#2022-02-0217:42wilkerluciowriting a function to convert one to another is trivial (except on the case of env usage, then it needs to figure the alternative on Pathom 3)#2022-02-0217:43imreThank you for the summary!#2022-02-0222:20wilkerlucio@alex.sheluchin fixed on main!#2022-02-0300:36KevinKHi, I’m wondering if this is expected behavior of priority. Here’s an example:
(-> {}
      (pci/register [@(pco/defresolver nested-single-attr [{in :input}]
                        {::pco/output [{:output [:sub-out-1]}]
                         ::pco/priority 1}
                        (prn "called single")
                        {:output {:sub-out-1 (format "(single-sub-1 %s)" in)}})
                     @(pco/defresolver nested-extra-attrs [{in :input}]
                        {::pco/output [{:output [:sub-out-1
                                                 :sub-out-2]}]}
                        (prn "called double")
                        {:output {:sub-out-1 (format "(double-sub-1 %s)" in)
                                  :sub-out-2 (format "(double-sub-2 %s)" in)}})])
      (p.eql/process [{[:input "my-input"]
                       [{:output [:sub-out-1
                                  :sub-out-2]}]}]))
returns
{[:input "my-input"]
 {:output
  {:sub-out-1 "(double-sub-1 my-input)", :sub-out-2 "(double-sub-2 my-input)"}}}
always, but when nested-single-attr has higher priority than nested-extra-attrs , it always prints ā€œcalled singleā€ and ā€œcalled doubleā€. Otherwise, it only prints ā€œcalled doubleā€. I’d hope there’d be a configuration so it only prints ā€œcalled doubleā€. But it seems like resolution happens individually for the nested attributes i.e. it selects a path for :sub-out-1 and then selects for :sub-out-2 if the resolver for :sub-out-1 didn’t already resolve :sub-out-2. I’d hope it would determine earlier that ā€œcostā€ of the whole query would be more for calling nested-single-attr first because it would imply an additional call to nested-extra-attrs.
#2022-02-0300:37KevinKMaybe this is a topic for https://github.com/wilkerlucio/pathom3/discussions/57? but it might be something different entirely#2022-02-0303:27wilkerluciothis is an interesting case, Pathom will try its best to fulfill the demand, so in this case, when single has priority, its called first, but there still the need for :sub-out-2, which will cause the other resolver to get called as well. its arguable if Pathom should optimize in this case, for example, imagine if nested-extra-attrs is some sort of cache that's a fallback for :sub-out-1, in this scenario is desirable to still call both resolvers.#2022-02-0303:28wilkerlucioI don't really like the current priority system, as described in the discussion you pointed, it doesn't work that well#2022-02-0303:28wilkerlucioI plan to get the default priorty to the same way pathom 2 did, based on path cost (which is dynamically computed as resolvers run)#2022-02-0303:29wilkerlucioI hope some better ideas may come up in the future to deal with manual priority#2022-02-0318:55KevinKCould one get a cost estimation at planning time? Like in this case, it’d be nice if the planner could see that the query requires both resolvers and sum of cost of nested-single-attr and nested-extra-attrs and compare it to the alternate of just calling nested-extra-attrs . I understand the cache argument being applicable when the outputs specified in the two resolvers are the same, and at runtime the prioritized resolver doesn’t return one of the attrs so it falls back to calling another. But in this case, it should know ahead of time that the latter resolver will need to be called regardless since the specified output of the first resolver doesn’t satisfy the query.#2022-02-0320:29wilkerlucioit is possible to calculate the cost at planning time, one caviat to be careful is because plans in pathom 3 are expected to be cached, and since the cost is dynamic, we need to be sure that's a separate step, the simplest solution is to always do that at running time#2022-02-0320:30wilkerlucioalso, priority never modifies the plan, the plan is always the same, the priority is about which branch from an OR node (which indicates alternatives for something) should be taken first#2022-02-0320:33wilkerluciothat said, Pathom 3 priority algorithm is plugabble, you can write a custom one and test some alternatives#2022-02-0320:02imreDoes anyone know what would be the equivalent or closest approximation of pathom2's ::p/wrap-parser in p3? Is it ::pcr/wrap-root-run-graph!?#2022-02-0320:34wilkerlucio:com.wsscode.pathom3.interface.eql/wrap-process-ast#2022-02-0320:35wilkerlucioI think this needs a new helper, I have made one for the viz connector, feels like this should be ported back to Pathom 3#2022-02-0320:35wilkerluciohere is the ws connector code you can copy to do that:#2022-02-0320:36wilkerluciohttps://github.com/wilkerlucio/pathom-viz-connector/blob/master/src/com/wsscode/pathom/viz/ws_connector/pathom3.cljc#L43-L60#2022-02-0320:48imrethanks a million!#2022-02-0322:02roklenarcicin pathom 2 what’s teh difference between parser and async-parser in terms of resolver returns? if I use async, must i return chan? can I return chan?#2022-02-0401:20souenzzoin pathom2, when using an async or parallel parser, your resolvers need to return: • or something that satisfies chan? https://github.com/wilkerlucio/wsscode-async/blob/master/src/com/wsscode/async/async_cljs.cljs#L7 • or something that is a map? We can confirm this information on this line of code: https://github.com/wilkerlucio/pathom/blob/main/src/com/wsscode/pathom/parser.cljc#L282#2022-02-0415:36roklenarcicI assume that returning promesa promise won’t work though#2022-02-0500:19souenzzonope. promesa is only supported in pathom3 and I'm not sure how it works exactly#2022-02-0722:58wilkerlucioin Pathom 3 everything is driven by Promesa promises, they proved to be much more efficient then core.async for the particular way Pathom processes things, except for the parallel processor which does use core.async for queuing#2022-02-1018:14roklenarciccore async is a more general tool, channels with queues are a very general approach to async… promises are a special case. For most async flows if you don’t need queues I assume that CompletableFuture that is wrapped by promesa should be more efficient. Did you test performance @U066U8JQJ#2022-02-1018:15wilkerlucioyes, performance using Promises is far superior than using promise-chan, I guess specially the part of the error handling/error propagation#2022-02-1018:16wilkerlucioI tried for a long time to just use core.async for everything, but after the experience with Promesa I know see two distinct concepts, for single-hit async processes is way better to use a focused tool like Promesa (CompletableFuture), I still like to use core.async when I need an async queue, but for single hit I'll always pick Promises#2022-02-0322:45imreIs there an equivalent to p2's ::pp/key-process-timeout in p3?#2022-02-0413:03wilkerluciono, did you find a situation with some sort locking going on?#2022-02-0814:47imreOops, sorry I forgot to respond to this. No, I was converting some test cases and it appears this key was used to set some sort of a global timeout to abort an async resolver that too long to produce a result#2022-02-0814:49wilkerluciothe reason this was created initially was more about not letting pathom screw itself, was to prevent hard to find locksteps in the process (usually do to Pathom's itself falt)#2022-02-0814:50wilkerluciofor Pathom 3, so far I prefer to leave this in the users hands, one way to do something similar is to write a wrap-resolve plugin, and for any async thing returned, wrap that in a timeout#2022-02-0814:50wilkerluciothat said, parallel process still new to Pathom 3 and more info/use cases can better tell the direction to handle these kinds of problems, feedback is much appreciated#2022-02-0814:52imreThank you. For now this was just a quick spike but we have plans to go ahead with the conversion so I'm sure we'll have feedback sooner or later#2022-02-0814:30sheluchinTo make the https://blog.wsscode.com/pathom/v2/pathom/2.2.0/connect/exploration.html#_setting_up_the_index_explorer_resolver in Pathom3, is it just: (get env ::pc/indexes) -> (get env ::pci/indexes)?#2022-02-0814:45wilkerlucioin Pathom 3 you just need to use the boundary interface (https://pathom3.wsscode.com/docs/eql/#boundary-interface), and it sets up the index explorer support automatically, or using the pathom connector, which also makes it automatically#2022-02-0814:46wilkerlucioif you are trying to support on some HTTP API, also make sure you setup the encoding to use transit, and use the transit readers/writers from pathom, so it can encode/decode resolvers and mutations#2022-02-0814:46wilkerluciothis tutorial covers the full http setup: https://pathom3.wsscode.com/docs/tutorials/serverless-pathom-gcf#2022-02-0817:42sheluchinHmm, I'm just trying to upgrade my Fulcro to use Pathom3. Everything seems to be working, except EQL autocomplete and the index explorer. I get this error:
I 2022-02-08T17:41:46.254Z                       _rad.pathom-common:- 28 - Request:  [{[:com.wsscode.pathom.viz.index-explorer/id [:fulcro.inspect.core/app-uuid #uuid "85fd9b5b-0a33-4c9e-8ded-2a0dd2f4d4a6" :remote]] [:com.wsscode.pathom.viz.index-explorer/id :com.wsscode.pathom.viz.index-explorer/index]}]
E 2022-02-08T17:41:46.256Z                             _rad.pathom3:- 31 - EQL query for :com.wsscode.pathom.viz.index-explorer/index cannot be resolved. Is it spelled correctly? Pathom error: {:com.wsscode.pathom3.error/cause :com.wsscode.pathom3.error/attribute-unreachable}
I 2022-02-08T17:41:46.257Z                       _rad.pathom-common:- 28 - Response:  {[:com.wsscode.pathom.viz.index-explorer/id [:fulcro.inspect.core/app-uuid #uuid "85fd9b5b-0a33-4c9e-8ded-2a0dd2f4d4a6" :remote]] {:com.wsscode.pathom.viz.index-explorer/id [:fulcro.inspect.core/app-uuid #uuid "85fd9b5b-0a33-4c9e-8ded-2a0dd2f4d4a6" :remote]}}
My code is based on the https://github.com/fulcrologic/fulcro-rad-demo/blob/1d8b7b1bba31aca925335f35f8b1486c6a25d4a0/src/xtdb/com/example/components/parser.clj#L29 and that call to pathom3/new-processor does use the boundary interface: https://github.com/fulcrologic/fulcro-rad/blob/fulcro-rad-1.1.6/src/main/com/fulcrologic/rad/pathom3.clj#L112
#2022-02-0817:46wilkerlucioare you trying on Pathom viz or Fulcro Inspect?#2022-02-0817:46wilkerluciobecause Fulcro Inspect doesn't support Pathom 3 at this time#2022-02-0817:47sheluchinAh! That makes sense. I was trying to use Inspect. I'll try setting up Viz.#2022-02-0820:00sheluchin@U066U8JQJ I'm not sure how to get Viz working with the rad demo code. I still need to use the connector, right? If so, where do I use it in the parser setup? Is this something I need to read the plugins page to understand? Haven't got to that one yet..#2022-02-0820:03wilkerluciothe config should be very similar to the JVM one, Pathom 3 doesn't have the concept of a parser, so its about configuring the env when defining it#2022-02-0820:04wilkerluciothe connector will do all the nescessary extensions (like exposing the indexes), so it should work strait from there, maybe its easy for you to try setting up it first without fulcro, and them placing it there (after all, its a differnet wrapping the call to the eql processor)#2022-02-0820:04wilkerlucioI'm assuming in this case you are making a CLJS Pathom configuration, right?#2022-02-0820:07sheluchinI've got it working in the GraphQL GitHub demo. That was my first step, and now I'm trying to use it with my Fulcro project. This is for the backend parser to connect to my database, as shown here: https://github.com/fulcrologic/fulcro-rad-demo/blob/1d8b7b1bba31aca925335f35f8b1486c6a25d4a0/src/xtdb/com/example/components/parser.clj#L22-L29#2022-02-0820:08sheluchinIt's that call to RAD's pathom3/new-processor that registers the indexes, which is where I set up the connector in the GraphQL repo, but don't know how to inject it into the logic here.#2022-02-0913:11wilkerlucioI dont have a lot of contexto on the RAD implementation, at some place you should be able to change env things#2022-02-0913:13sheluchinThanks @U066U8JQJ but it's okay. I don't want to distract you from your work too much if the answer isn't something obvious. If I figure it out I will commit some docs somewhere to explain it.#2022-02-0913:15sheluchinNot the end of the world. It's alpha stage and I wanted to provide feedback around friction points I'm personally seeing, but I still have a lot to learn here so take it with a grain of salt šŸ™‚#2022-02-0913:26wilkerluciono worries, I appreciate all the feedback, better to know the pain points šŸ™‚#2022-02-0912:11sheluchinJust a heads up that the link to Execution Nodes https://pathom3.wsscode.com/docs/environment#comwsscodepathom3connectplannernode is not currently valid.#2022-02-1008:16StefanHi all! I’m investigating whether we want to start using Pathom. I’ve read some documentation, seen videos, and listened to the podcast episodes with Jacek Schae. I’m sold on the concept, no worries there šŸ™‚ I guess from what I’m reading here that we should stick with Pathom 2 for now, as this is an existing production system. There is one area that I’m not clear about: the client-server story. I’m envisioning a situation where our (ClojureScript/react) client uses Pathom to get its data from our (Clojure/Ring) server. Suppose I start doing that, I will have to build a lot of resolvers on the server and I can of course start using Pathom server-side. But how do I use those same resolvers from the client? I guess that I would need the same set of resolvers in terms of input and output specification, but instead of going to the database, they have to go to our backend. How is this supposed to work? Am I missing some part? Even better, is there maybe an example of such a setup somewhere? Many thanks for your thoughts!#2022-02-1012:10Piotr RoterskiHey @UGNFXV1FA šŸ‘‹ Happy to hear you’re sold on the concept! AFAIU the idea for client-server communication is that the client sends EQL query over to server which handles the request using pathom parser to resolve the query and return the response data. The most popular pathom setup is its integration with fulcro and you can see how it all works together in https://github.com/fulcrologic/fulcro-rad-demo. Granted you don’t need to adopt the entire framework to use pathom, you can still take a look at fulcro’s code to see how this client-server communication is set up there. Here are some pointers: • https://github.com/fulcrologic/fulcro/blob/develop/src/main/com/fulcrologic/fulcro/networking/http_remote.cljs#L356 to the server (there’s a lot more going on there, but https://github.com/fulcrologic/fulcro-rad/blob/develop/src/main/com/fulcrologic/rad/application.cljc#L90) • server uses ring https://github.com/fulcrologic/fulcro-rad-demo/blob/develop/src/shared/com/example/components/ring_middleware.clj#L37-L38`/api` handler which passes the query to the https://github.com/fulcrologic/fulcro-rad-demo/blob/develop/src/xtdb/com/example/components/parser.clj#L33 There’s a lot of fulcro and fulcro-rad convenience wrapping around all of it, but if you follow the trail and skip what you don’t need, the basic idea is simple.#2022-02-1012:15StefanThanks for the pointers Piotr; indeed I was not planning on switching to fulcro, but I will definitely investigate the pointers that you gave, I’m sure that will help!#2022-02-1013:31jmayaalvAlso we have been using pathom3 in prod for a few months without any major issue.#2022-02-1013:32jmayaalvSo unless you have something very specific that is only supported in pathom2 i would stick with pathom3#2022-02-1014:34plinsI would advocate for pathom3 as well I had some problems with pathom2 and nested batched resolvers, those problems are solved in pathom3 now#2022-02-1016:13markaddlemanI'll throw my two cents in: We've been using pathom3 for several months. We have not had any major problems.#2022-02-1017:19wilkerluciohello @UGNFXV1FA, at this point I also suggest starting with Pathom 3, performance is better and it has more features, in terms of stability, the basic features are stable, the parts that I don't consider stable are: • parallel process: feel free to try, but keep in mind the chances of erros are higher, that said I always appreciate bug reports so we can start closing the gap • distributed process: that's a feature you can connect multiple Pathom instances in a federated way, this is a killer feature that I want to get more stable, this allows you for example to have one pathom running on the server, and another instance running on the client, so the client connects to the server (supporting everything there) while is also capable of adding its own resolvers that will run in the client side, no current known bugs but I feel it needs more usage to find problems (specially when combining more complicated features like optional inputs and nested inputs)#2022-02-1017:21wilkerlucioa heads up that goes for also everybody else here, I've been working to improve the error handling in the strict mode (so we can render partial complete plans on Pathom Viz for example), this will cause minor breaks on the internals (I believe no user will be affected by those), and one that may affect you all is a change I'm thinking for the boundary interface, I want to change it to never throw (even on strict mode), instead it should return an error in a data format, I'm going over this because sending exceptions over the wire is a pain and I think a data format will smooth things out#2022-02-1018:25markaddlemanI love this idea - I'm particularly happy that partial plans will be available in the vizualizer#2022-02-1019:12StefanThat's great feedback from all of you, awesome! I guess it will be Pathom 3 then, I'm glad it's already stable enough! And @U066U8JQJ it sounds like that federation is what I was hoping for, sounds great! šŸ˜€#2022-02-1017:21wilkerlucioa heads up that goes for also everybody else here, I've been working to improve the error handling in the strict mode (so we can render partial complete plans on Pathom Viz for example), this will cause minor breaks on the internals (I believe no user will be affected by those), and one that may affect you all is a change I'm thinking for the boundary interface, I want to change it to never throw (even on strict mode), instead it should return an error in a data format, I'm going over this because sending exceptions over the wire is a pain and I think a data format will smooth things out#2022-02-1114:26markaddlemanI'm using Pathom on the server-side to help organize the code that generates large and somewhat complex SQL for an analytics application. I have resolvers that generate filter expressions, aggregations, joins etc. In many cases it takes several hundred milliseconds to generate the SQL. In the common case, the structure of the SQL remains the same only the particular values change - ie, this is a perfect case for prepared statements and query variables. What I think I want to do is, continue to use Pathom to generate the SQL but, rather than the input to resolvers being actual values, I want the input to be the path to the values. The application would generate a HoneySQL-like data structure but with paths for values. At the end of processing, I'd have a resolver replace the paths with actual values. This way, I can use Pathom's caching system to cache the HoneySQL data structure thus saving several hundred milliseconds (in most cases). My question is this: Does this make any sense? Is there a way for a resolver to get its inputs as paths rather than values?#2022-02-1115:10sheluchinI'm curious about your implementation. Do you happen to have any of the source available?#2022-02-1115:30wilkerlucioI feel like getting the paths is not really a thing here, because Pathom processes one entity at a time, and its about flowing values though the resolvers. for nested inputs it kind just goes and modifies the queries for deeper values, but its kind like they were flat for your description seems like you could try to do something by processing the plan instead of doing stuff at the run step dynamic resolvers also seem to fit here, in this case its about separating the parts of your query that are related to SQL vs parts that you will process with pathom resolvers, with dynamic resolvers you can leverage Pathom capabilities to isolate the SQL parts (by making them a single dynamic resolver), then in the dynamic resolver you gonna receive just the SQL query parts, and them you can process on top of it#2022-02-1116:14markaddleman@UPWHQK562 Unfortunately, this is proprietary code and I can't share it directly but I'm happy to answer your questions. In fact, I have been meaning to write a blog article on how we're using Pathom so answering your questions would be helpful for me!#2022-02-1116:20markaddleman@U066U8JQJ - Thanks for the pointer. Processing the plan does seem like a good idea. Can you point me to the API to obtain the plan without calling process ? Or, do you think I should invoke using process and use an interceptor to skip resolvers if appropriate? Your idea for dynamic resolvers also seems really interesting. I need to study up on dynamic resolvers to understand the details. I'm sure I'll have lots of questions as I move forward.#2022-02-1116:20markaddlemanThanks!#2022-02-1207:59myguidingstar@U2845S9KL that kind of prepared statements is actually what's already available with https://walkable.gitlab.io though I didn't have time to write the doc for that (and for many other things). Besides capturing values in closures, you can leverage walkable's ability to generate in advance as much as possible the sql string from data structure. For instance, if you have this data structure [:and [:< 1 'foo-var] [:< 'bar-var 10]], walkable can generate this string ((? < 1) AND (? < 10)) and you will only need to provide values for the vars (for instance {'foo-var 3, 'bar-var 7} in this case) to have it realized automatically to ((3 < 1) AND (7 < 10)).#2022-02-1222:38markaddlemanThanks for the pointer to walkable! I looked at walkable several months ago and I did not think it suited my use case but now I understand my use case better and I can ask specific questions. I'll post to take this conversation over to #walkable#2022-02-1419:50wilkerluciohello everybody! there is some progress on the strict error story! Pathom Viz just got a new release today with some needed changes to support this new error handling, if you already have Pathom Viz, just close and open it again to get the update done automatically (ensure you have version 2022.2.14). For the Pathom 3 side of things, I open a PR to get some feedback before merging the changes, please see the details at https://github.com/wilkerlucio/pathom3/pull/129#2022-02-1509:42StefanQuick question (to @wilkerlucio probably): Is https://github.com/wilkerlucio/pathom3-datomic also ready for production use, or should we stay away for now?#2022-02-1514:10wilkerlucioI suggest avoiding it, the problem is more about access control, so in the end its probably better to write manual resolvers to have a tight grip on it, this is how Fulcro users do with Datomic#2022-02-1516:58wilkerlucio#2022-02-1619:37markaddlemanMy regression tests pass with this snapshot. Every client query goes through a lenient mode phase followed by a strict mode phase (for silly, backward compatibility reasons). In the lenient mode, I expect a lot of the query attributes to be unresolvable. The result of the lenient mode query is massaged a little and then executed as a strict query. I do not expect any resolver to throw an exception. The upshot is that both phases of the query processing is working without a problem. I plan to start developing using this snapshot (and the new visualizer) and I'll see how exceptions are handled.#2022-02-1620:05markaddlemanI meant to put this comment under https://clojurians.slack.com/archives/C87NB2CFN/p1644868244592199#2022-02-1714:08HukkaDoes Pathom have any concept of different potential ways to get from A to B? Say for example that A is a Product, that has manufacturer, id, and name. We want to know the cost. For some products we have a database from names to costs. But for some other products we need to query the manufacturer's system with the id to get the cost, and with some we just cannot get the cost in any way.#2022-02-1714:45markaddlemanYes. Our app uses this. It's not a specific feature, per se. It's simply an outcome of how Pathom works.#2022-02-1714:46markaddlemanIn a nutshell, "resolvers" declare their inputs and their outputs.#2022-02-1714:46markaddlemanA client makes a query for a particular output and Pathom computes a "plan" to obtain the result. The plan can have many branches reflecting all the possible ways that the result could be computed given the input from the client's query.#2022-02-1714:47markaddlemanI think it's useful to think of Pathom as automated, dynamic function composition.#2022-02-1714:53Ivanafaik, this goes a bit further in the sense that the choice of which path will be followed first is based on calculating a cost for that data access#2022-02-1715:26HukkaSure, but how would I describe that one resolver needs name, outputs cost. Second needs manufacturer and id, outputs cost. But neither might, or might not be able to produce the cost, so both need to be tried, in a given order.#2022-02-1715:26HukkaI could of course make one resolver that takes all three as input, but that wouldn't work in cases where for example the name is not available#2022-02-1715:35markaddlemanI'm not aware of cost-based planning. Pathom uses a sort mechanism (I believe you could create a dynamic sorting capability which could mimic cost based sorting but this would require extending Pathom)#2022-02-1715:37markaddleman@U8ZQ1J1RR I have almost exactly your use case. Suppose you have two resolvers that can product cost. You can provide Pathom the order in which the resolvers should be tried using the priority system. If the first resolver cannot compute cost, it simply returns nil. Pathom will automatically try the second resolver.#2022-02-1715:50HukkaGreat. Is it just the order which the index is created? Is the seq accessed in from first to last, last to first or pop order (so vectors and lists work differently)?#2022-02-1716:07markaddlemanThe priority is set explicitly in the resolver https://pathom3.wsscode.com/docs/resolvers/#prioritization#2022-02-1716:50HukkaOk, so return ::pco/unknown-value, from the higher priority resolver and Pathom will try the next one. At least as long as the paths are equal length (the discussion seems to show that things get confused in other cases)#2022-02-1716:54markaddlemanYes, that's correct. In practice, I generally write mutually exclusive guarding if statements on each resolver so I don't have to worry about path length.#2022-02-1718:02mauricio.szabo@U8ZQ1J1RR I also have the same situation at an open-source project, but honestly, if you do need to follow a specific order, the "default priorization" of Pathom does not help too much, honestly - it easily gets into some weird cases when it does not what you expect. More info here: https://github.com/wilkerlucio/pathom3/discussions/57#2022-02-1718:03HukkaThanks, yeah. That's the discussion I was referring to, and the path length differences.#2022-02-1718:07mauricio.szaboI just posted how I kinda solved the issue on the discussion now, if you want to check šŸ™‚#2022-02-1721:10wilkerluciothe current prioritization is flacky as @U3Y18N0UC, and just to be clear we are talking Pathom 3, Pathom 2 by default uses a prioritization based on "path weight", to do this pathom tracks the time that resolvers are taking to run (dynamically, it updates this value every time a resolver gets called), so it will choose the path that has the least weight (for example, you may have a path with 5 resolvers that take 3ms each, vs a path with 1 resolver that takes 100ms, in this case the path with 5 resolvers still better)#2022-02-1721:12wilkerlucioI'm planning to bring that back to Pathom 3 and make that the default prioritization mechanism again, that said, this part of the algorithm is extensible, so you can plug you own prioritization function if you want (like Mauricio did). and I'm open for other general ideas to make prioritization work, I didn't had a chance to look closer in the latest mauricio one, but it could be that šŸ™‚#2022-02-1721:12wilkerlucio@U3Y18N0UC do you think the one you are using is general enough to be part of the library? what you see as limitations of it at this point?
#2022-02-1721:19mauricio.szabo@U066U8JQJ the only limitation that I see is that I didn't battle-test it yet. It seems to be working fine in my case, and I'm not seeing anything weird happening so far, and the code is generic enough. But it was mostly a "let's try this, hey, it works!" than "let's analyze all possible cases" šŸ™‚#2022-02-1721:22mauricio.szaboThe code is literally "let me get the keys that I expect from the or-node", then "let me get all successors that provide these keys", and them sort by the one with maximum priority#2022-02-2500:50wilkerlucio@U8ZQ1J1RR the algorithm from@U3Y18N0UC just landed on main in Pathom 3! I'll cut a new release including it tomorrow (its the new default, I replaced the old one with it)#2022-02-2500:51wilkerluciothanks @U3Y18N0UC!#2022-02-2514:27mauricio.szaboWow, glad to see that it worked! šŸ™‚#2022-10-1711:37Eric DvorsakI think the issue I mentioned above might be related to this thread. I'll dive into the code to see if there's a small fix. Essentially the issue is that in the absence of any priority OR branches the sorting doesn't prioritize the shortest path#2022-10-1712:07Eric Dvorsakok so I found the solution. The weight of the branch isn't taken into account unless one registers the resolver-weight-tracker. When it is not there, the first item in the set returned by the prioritization is used. I think that a small improvement would be to still take weight into account, with a default of 1 for each node when there is multiple candidates returned by the prioritization#2022-10-1711:37Eric DvorsakI think the issue I mentioned above might be related to this thread. I'll dive into the code to see if there's a small fix. Essentially the issue is that in the absence of any priority OR branches the sorting doesn't prioritize the shortest path#2022-02-2113:13sheluchinHow do you do the GraphQL ... on Type thing in EQL? Trying to translate this query for Pathom:
{
  repository(name: "pathom3", owner: "wilkerlucio") {
    object(oid: "8e0a0453189bc13d045944d128248d0f2e7c5c17") {
      id
      commitUrl
      ... on Commit {
        committedDate
      }
    }
  }
}
#2022-02-2113:21wilkerluciojust ask for the attribute (in this case would be something like :ns.Commit/commitedDate, Pathom adds the ... on part automatically#2022-02-2113:22sheluchinlol it's always simpler than I first guess. Thanks!#2022-02-2113:24wilkerluciothis is an advantage of knowing the full context of any attribute every time šŸ™‚#2022-02-2113:25wilkerluciolet me know if you find any issues with it#2022-02-2113:43sheluchinWill do. So far so good.#2022-02-2115:29sheluchinIs there anything built in or a suggested approach for handling GraphQL pagination? Similar approach as in the https://pathom3.wsscode.com/docs/tutorials/hacker-news-scraper/#traversing-pagination?#2022-02-2118:03wilkerlucionothing pre-baked, you have to do your own on this#2022-02-2120:49wilkerlucio#2022-02-2121:14sheluchinPulling out values from a GraphQL result with much nesting seems like a lot of work. Example:
[{:github.Organization/login login}
   ['({:github.Organization/repositories
       [:github.RepositoryConnection/totalCount
        {:github.RepositoryConnection/edges
         [{:github.RepositoryEdge/node
           [:github.Repository/id
            :github.Repository/name
            {:github.Repository/issues
             [:github.IssueConnection/totalCount]}
            {:github.Repository/pullRequests
             [:github.PullRequestConnection/totalCount]}
            {:github.Repository/defaultBranchRef
             [:github.Ref/name
              {:github.Ref/target
               [{:github.Commit/history
                 [:github.CommitHistoryConnection/totalCount]}]}]}]}]}]}
      {:first 2})]])
Result:
{:github.Organization/repositories {:github.RepositoryConnection/totalCount 29
                                         :github.RepositoryConnection/edges [{:github.RepositoryEdge/node {:github.Repository/id "MDEwOlJlcG9zaXRvcnk5NzAzMjE1OA=="
                                                                                                           :github.Repository/name "fulcro"
                                                                                                           :github.Repository/issues {:github.IssueConnection/totalCount 270}
                                                                                                           :github.Repository/pullRequests {:github.PullRequestConnection/totalCount 233}
                                                                                                           :github.Repository/defaultBranchRef {:github.Ref/name "develop"
                                                                                                                                                :github.Ref/target {:github.Commit/history {:github.CommitHistoryConnection/totalCount 4436}}}}}
                                                                             {:github.RepositoryEdge/node {:github.Repository/id "MDEwOlJlcG9zaXRvcnk5NzAzNjU0Nw=="
                                                                                                           :github.Repository/name "fulcro-spec"
                                                                                                           :github.Repository/issues {:github.IssueConnection/totalCount 16}
                                                                                                           :github.Repository/pullRequests {:github.PullRequestConnection/totalCount 7}
                                                                                                           :github.Repository/defaultBranchRef {:github.Ref/name "develop"
                                                                                                                                                :github.Ref/target {:github.Commit/history {:github.CommitHistoryConnection/totalCount 709}}}}}]}}
Is there any sugar for making life easier here? I was thinking of using https://github.com/escherize/tracks as a helper.
#2022-02-2121:28sheluchinOr maybe just providing a detailed ::pco/output is enough? :thinking_face:#2022-02-2122:05Bjƶrn EbbinghausMaybe you could add a resolver, that connects Ė‹repositoriesĖ‹ directly to the repository? Input repositories, output repository id Then you could skip edges and node#2022-02-2200:54wilkerluciothis is more about GraphQL design for pagination, which adds these extra steps, like @U4VT24ZM3 said, you may be able to shortcut by writing resolvers that simplify the request. At this point I think we need to verify if we can properly forward the params in this situation#2022-02-2201:04wilkerlucioI think this falls in the same domain as this issue, but instead of forwarding sub-queries parts, we may want something to point how to forward params: https://github.com/wilkerlucio/pathom3/issues/84#2022-02-2210:53sheluchinThanks for the input. A shortcut resolver seems like it makes the most sense right now, but it does feel like there is potential for a shorter description.#2022-02-2217:34sheluchinI'm not sure I understand the shortcut resolver approach. Is the idea that I should: • create a resolver that takes :gh.Organization/repositories • have that resolver do the work requesting additional levels down (edges and nodes, down to :Repository/id), and flattening and return the results Could someone please elaborate or link me to some source or documentation with a relevant example?#2022-02-2200:58wilkerlucio#2022-02-2210:55sheluchinHow do I provide a GraphQL argument like this one?
issues(states: OPEN) {
  totalCount
}
[IssueState!].OPEN

An issue that is still open
#2022-02-2212:42Bjƶrn EbbinghausIt should be:
{(:issues {:states "OPEN"})
 [:totalCount]}
But yes. The pathom3 docs doesn't show GraphQL parameters.
#2022-02-2212:43Bjƶrn EbbinghausHm… Or is "OPEN" some sort of Enum Value… In this case… I don't know. Maybe as a keyword? :OPEN#2022-02-2213:18sheluchinTried both of those and it complains that the filter argument isn't an IssueState!.#2022-02-2213:23sheluchin
"clojure.lang.ExceptionInfo: Resolver github/organization-ident-entry-resolver exception at path []: GraphQL Error: Argument 'states' on Field 'issues' has an invalid value (\"OPEN\"). Expected type '[IssueState!]'. at path [\"query\" \"organization\" \"repositories\" \"edges\" \"node\" \"issues\" \"states\"] #:com.wsscode.pathom3.path{:path []}"
It looks like it encodes both "OPEN" and :OPEN as "OPEN" in the request it sends anyhow.
#2022-02-2213:26Bjƶrn Ebbinghaus"IssueState.OPEN"? :IssueState/OPEN?#2022-02-2213:32sheluchinNo luck on any of those.#2022-02-2213:38sheluchinThe IssueConnection can take a few different arguments. Among them:
states: [IssueState!]

A list of states to filter the issues by.

labels: [String!]

A list of label names to filter the pull requests by.
So there's a difference in arg type and it looks like our guesses for the notation are not correct.
#2022-02-2213:39sheluchinPathom Viz didn't really offer any hints on this point either, and IssueState wasn't discoverable in the Explorer.#2022-02-2214:09Bjƶrn EbbinghausHm... The https://github.com/wilkerlucio/pathom3-graphql/blob/main/src/main/com/wsscode/pathom3/graphql.cljc#L28doesn't even query for enum values. Maybe there is simply no enum support in Pathom yet? @U066U8JQJ should know. šŸ™‚#2022-02-2214:11sheluchinI think that's likely it. This stuff is fairly new so I'm just flagging any issues I run into šŸ™‚ don't mind me!#2022-02-2214:11Bjƶrn EbbinghausBut even if it doesn't query for the enum values... Pathom should know the IssueState type..#2022-02-2214:12wilkerlucioin the enum case you should use a symbol#2022-02-2214:12wilkerlucioas (:issues {:state OPEN}#2022-02-2214:14sheluchinhahaha yet again it's always simpler than my first ten guesses#2022-02-2214:14sheluchinTY, that does indeed work.#2022-02-2516:32sheluchinMoved to https://github.com/wilkerlucio/pathom3-graphql/issues/15#2022-02-2517:09wilkerlucioI need to investigate that, can you open an issue on pathom3-graphql repo to keep it tracked?#2022-02-2517:10sheluchin@U066U8JQJ sure thing. I'll just copy the above over into there. Thank you šŸ™#2022-02-2518:54wilkerlucio#2022-02-2522:29kendall.buchananThanks so much, @U066U8JQJ, for maintaining and documenting such a fantastic library. We’re about to go live with a Pathom3 implementation, and we’ve really loved it.#2022-02-2521:45markaddlemanWith 2022.02-25-alpha , it looks like if I have two resolvers outputting the same attribute where one has declared ::pco/priority and the other does not, the one without ::pco/priority is not invoked. Is that the intended behavior?#2022-02-2610:35markaddlemanTo clarify, the resolver with ::pco/priority returns nil so I expected the resolver without ::pco/priority to be invoked#2022-02-2617:49markaddlemanActually, I got this backwards: The resolver without ::pco/priority is always invoked while the resolver with ::pco/priority is not invoked#2022-03-0317:45wilkerluciohello mark, can you make a repro example please?#2022-03-0318:17markaddlemanSure#2022-03-0419:47wilkerlucioI tried to reproduce with this:
(ns com.wsscode.pathom3.demos.repro-priority
  (:require [com.wsscode.pathom3.connect.operation :as pco]
            [com.wsscode.pathom3.connect.indexes :as pci]
            [com.wsscode.pathom3.interface.eql :as p.eql]))

(pco/defresolver a1 []
  {:a 1})

(pco/defresolver a2 []
  {::pco/priority 1
   ::pco/output [:a]}
  nil)

(def env
  (pci/register [a1 a2]))

(comment
  (p.eql/process env [:a]))
#2022-03-0419:47wilkerluciobut in this case I get the desired result, so I wonder what kind of edge case you may have hit#2022-03-0419:49markaddlemanI'll try to reproduce this weekend#2022-03-0512:10markaddlemanI can't reproduce. šŸ˜• If I run into it again, I'll be sure to create a repro case before I forget all the context#2022-03-0517:40wilkerluciono worries, it will eventually bite back again#2022-03-0517:42wilkerluciofor a contexto to help debug: the main change in the algorithm is that the previous path would look at every node in the subpaths to find the highest priority, while this new one will look only at nodes that are responsible directly for the attribute that the OR node is looking for#2022-02-2822:14hjrnunesHi! Can someone riddle me this, please? Why can I get to name in the last query but not in the first one? Thanks!
(require '[com.wsscode.pathom3.connect.built-in.resolvers :as pbir])
(require '[com.wsscode.pathom3.connect.indexes :as pci])
(require '[com.wsscode.pathom3.connect.operation :as pco])
(require '[com.wsscode.pathom3.interface.eql :as p.eql])

(def mock-accounts [#:gnucash.account{:id   "e8bb8cdcdd964239bc0255c447f5174d",
                                      :name "Root Account"}])

(def mock-transactions [#:gnucash.transaction{:id          "75d3cf61239f425c9d1d5da30ef56476",
                                              :description "Starting balance",
                                              :splits      [#:gnucash.split{:id       "c5b722950d5947508d1acb41526ee278",
                                                                            :quantity "203995/100",
                                                                            :account  "e8bb8cdcdd964239bc0255c447f5174d"}
                                                            #:gnucash.split{:id       "b677a56fa6574df5bab5963d362ce0b6",
                                                                            :quantity "-203995/100",
                                                                            :account  "e8bb8cdcdd964239bc0255c447f5174d"}]}])


(pco/defresolver gnucash-accounts []
  {:gnucash/accounts mock-accounts})

(pco/defresolver gnucash-account [{:gnucash.account/keys [id]}]
  {::pco/output [:gnucash.account/id
                 :gnucash.account/name]}
  (->> mock-accounts
       (filter (comp #{id} :gnucash.account/id))
       (first)))

(pco/defresolver gnucash-transactions []
  {::pco/output [{:gnucash/transactions [:gnucash.transaction/id
                                         :gnucash.transaction/description
                                         {:gnucash.transaction/splits
                                          [:gnucash.split/id
                                           :gnucash.split/quantity
                                           :gnucash.split/account]}]}]}
  {:gnucash/transactions mock-transactions})

(pco/defresolver gnucash-transaction [{:gnucash.transaction/keys [id]}]
  {::pco/output [:gnucash.transaction/id
                 :gnucash.transaction/description
                 :gnucash.transaction/splits]}
  (->> mock-transactions
       (filter (comp #{id} :gnucash.transaction/id))
       (first)))

(pco/defresolver gnucash-transaction-splits [{:gnucash.transaction/keys [id]}]
  {::pco/input  [:gnucash.transaction/id]
   ::pco/output [:gnucash.split/id
                 :gnucash.split/quantity
                 :gnucash.split/account]}
  (->> mock-transactions
       (filter (comp #{id} :gnucash.transaction/id))
       (first)))

(def env
  (pci/register
    [gnucash-accounts
     gnucash-transactions
     gnucash-account
     gnucash-transaction
     gnucash-transaction-splits
     (pbir/alias-resolver :gnucash.account/parent :gnucash.account/id)
     (pbir/alias-resolver :gnucash.split/account :gnucash.account/id)]))


(p.eql/process env
               {:gnucash.transaction/id "75d3cf61239f425c9d1d5da30ef56476"}
               [:gnucash.transaction/description
                {:gnucash.transaction/splits [:gnucash.split/id
                                              :gnucash.split/quantity
                                              {:gnucash.split/account [:gnucash.account/name]}]}])

;; =>
;; #:gnucash.transaction{:description "Starting balance",
;;                       :splits [#:gnucash.split{:id "c5b722950d5947508d1acb41526ee278",
;;                                                :quantity "203995/100",
;;                                                :account "e8bb8cdcdd964239bc0255c447f5174d"} ;; <----- NO NAME!
;;                                #:gnucash.split{:id "b677a56fa6574df5bab5963d362ce0b6",
;;                                                :quantity "-203995/100",
;;                                                :account "e8bb8cdcdd964239bc0255c447f5174d"}]} ;; <----- NO NAME!


(p.eql/process env
               {:gnucash.split/account "e8bb8cdcdd964239bc0255c447f5174d"}
               [:gnucash.account/name])

;; =>
;; #:gnucash.account{:name "Root Account"}  ;; <----- HAS NAME!
#2022-02-2822:42nivekuiltry this query
(p.eql/process env
               {:gnucash.transaction/id "75d3cf61239f425c9d1d5da30ef56476"}
               [:gnucash.transaction/description
                {:gnucash.transaction/splits [:gnucash.split/id
                                              :gnucash.split/quantity
                                              :gnucash.split/account
                                              :gnucash.account/name]}])
#2022-02-2822:50hjrnunesHi, yes I can see the right :gnucash.account/name values. Thank you!#2022-02-2822:50hjrnunesBut I'm struggling to understand what's going on...#2022-02-2822:51hjrnunesWouldn't I have to join via :gnucash.account/id explicitly?#2022-02-2822:52nivekuilnope, your alias resolver is doing that mapping#2022-02-2822:53nivekuilif you remove your alias resolver you can see it won't work at all, because :gnucash.split/account needs to have as its value a map, not a string, for pathom to understand it as joinable#2022-02-2822:53nivekuilso :gnucash.split/account {:gnucash.account/id "e8bb8cdcdd964239bc0255c447f5174d"} instead of just the string ID#2022-02-2822:53hjrnunesAh that last one makes sense, yes#2022-02-2822:55hjrnunesThank you! That was really helpful!#2022-02-2822:55nivekuilno problem! surprised to see anyone using gnucash#2022-03-0108:37hjrnunesYeah! I use it for my personal accounting. But I'm trying to get some bayesian transaction categorising going, so I need to scan the gnucash database to compile word frequencies. I thought Pathom could be useful for denormalising it#2022-02-2822:46nivekuilwith the new way errors are done, it seems a call to p.eql/process that used to be an exception (`All paths from an OR node failed`) now tries to print/serialize the whole env , which in my case is so big it causes the JVM to crash. not sure where the env is being added but is there a way to hide it from the error result?#2022-03-0316:40jjttjjI have to second the request for the ability to hide the full env from the error info. Either that or it's been a reason to force me to be more conscious of what my tooling does with massive error data.#2022-03-0317:43wilkerluciohello folks, I cna surely consider some of those changes, but before I wonder if you guys considered using the boundary interface in this case?#2022-03-0317:43wilkerluciobecause the boundary interface will convert the errors in something less dreading ot send over the wire (that doesn't include full env)#2022-03-0317:44wilkerluciothe reasoning for the full env in the process error is to allow underlying usages to have full context about the problem#2022-03-0318:05jjttjjI hadn't used that yet but will look into it#2022-03-0319:26wilkerluciosorry, I said the wrong name, its boundary interface, docs here: https://pathom3.wsscode.com/docs/eql/#boundary-interface#2022-03-0319:49wilkerlucio@U797MAJ8M and to answer you directly, env is getting add as part of the exception data on strict errors#2022-03-0323:55nivekuilhave not looked at the boundary interface yet, but I was crashing the JVM just debugging stuff in the REPL so I think it would not have saved me#2022-03-0317:34jjttjjI have a list of things with references to different entities, but they are keyed with different keys. Is there a way in pathom3/eql to match any key and do a join if it's possible to?
;;; id->thing index

{1 {:name "A" :fk1 {:id 2}}
 2 {:name "A" :fk2 {:id 1}}
 3 {:name "A" :fk3 {:id 4}}
 4 {:name "b" :fk4 {:id 3}}}

;;; data i'm querying
{:things [{:name "A" :fk1 {:id 2}}
          {:name "A" :fk2 {:id 1}}
          {:name "A" :fk3 {:id 4}}
          {:name "b" :fk4 {:id 3}}]}

;;; I want a query something like this, look up the name for each entity pointed to by
;;; the fk
'{:things [{'* [:id :name]}]}
#2022-03-0317:51wilkerlucionot in an automatic way like that, because Pathom is design for lazyness, so a generic join is not a thing but you can make it explicity if you are able to know all possible foreign keys ahead of time, in this case you can make a common name to which all foreign options can converge to, something like:
(defn alias-join [source-name target-name]
  (pco/resolver (pbir/attr-alias-resolver-name source-name target-name "joined")
    {::pco/input  [source-name]
     ::pco/output [{target-name [:id :name]}]}
    (fn [_ input]
      (let [data (get input source-name)]
        {target-name data}))))

(pci/register
  [(alias-join :fk1 :generic-fk)
   (alias-join :fk2 :generic-fk)
   (alias-join :fk3 :generic-fk)
   (alias-join :fk4 :generic-fk)])

[{:things [{:generic-fk [:id :name]}]}]
#2022-03-0317:51wilkerlucionote that pathom needs to know the sub-query to work properly with nested queries (in case you have those)#2022-03-0317:52jjttjjAwesome, thanks for the help @U066U8JQJ / @U2J4FRT2T#2022-03-0317:50souenzzo@jjttjj one way to do that is create one alias pc/alias :fk1 :fk-any for each fk#2022-03-0322:34jjttjjI've been getting deep into pathom for a few days now and still getting a sense for where to draw the line between pathom and a (datomic-esque) database, particularly when it comes to exploring data. For data you already have, pathom quite ideal (or intended) for arbitrary exploration in the way that a full query language is. But if what you have is a REST api, or especially multiple related REST apis, pathom is pretty great for exploration compared to just using REST. Has anyone used anything like a plugin that saves all data you fetch with pathom to a db which could then be queried more flexibly? Or even just a high level workflow that has functions (fetch [eql]) + (q [datalog]) and alternating between the two in a repl session? Just trying to get a better sense in general for how pathom and a db relate. My quick google searches seem to show that libraries putting pathom on top of a db to enable eql queries seem more common than using pathom as a way to feed data to a db. But maybe that's just because the latter doesn't really need a library?#2022-03-0400:09lilactownsomething i've done is fetch a tree of data via pathom and then put it in a datascript db as a client side cache#2022-03-0400:10lilactownthis worked ok, but datascript is actually quite slow for a client side cache, which led me to build https://github.com/lilactown/pyramid#2022-03-0400:12lilactownpyramid doesn't really have good support (it's very alpha) for datalog style queries, but you can for example fetch some data from a pathom endpoint, put the result in a pyramid db, and then run the same EQL query on the pyramid db to avoid another round trip#2022-03-0400:13lilactownyou can also treat it like a normal map and just (get-in db ,,,) whatever you need#2022-03-0400:13lilactownif you like datalog tho, i would suggest datascript or perhaps asami#2022-03-0412:33wilkerluciogreat ask @jjttjj, I personally dont think that line is very clear, and I love to hear people finding interesting ways to use Pathom (like @U3Y18N0UC using it for REPL connections in Clover). I personally see pathom as this "big controller", that does coordination for you, in this view it makes sense to wrap things around pathom instead of otherwise, because this allows you to have the flexibility to change the implementation of your request, while keeping a consistent and evolvable interface (via the attribute names and their evolution), in this sense, the "need" (which is the shape) is a complete abstract definition, from which you could load via DB, via some mock data, or even via generators (I have actually done that, a pathom that just generates random data based on the specs of the attributes, to use during the rendering of Fulcro components in test mode, so we can try many different data variations without writing any specific one)#2022-03-0412:34wilkerluciosometimes you need more performance on queries (like on the UI), and in this case you may wanna dump the data somewhere else, like @U4YGF4NGM pointed out with Pyramid, I also see Fulcro as a variation of the same idea, having a local db separate from the "external sources", I think Fulcro has a great full stack story on that#2022-03-0816:33jjttjjSorry for the late reply but thanks for the responses! I've been using datascript as a default db, and have dabbled with all the other datalog things to varying degrees. The "big controller" thing is something I'm slowly grasping. On the surface it seems like pathom is something like a graphql replacement for specifying exactly what is needed for a page over a wire. It's really cool that it serves that role AND as a general computation engine. I've consdered doing things like:
(process {::start time1
            ::end   time2}
   [::user-signups
    {::posts [::title ::slug]}
    ::replies])
to grab any entities between two times. Still exploring and figuring things out but it's a fun process, and the way in which resolvers are built up feels useful even if ultimately the same functionality is moved elsewhere. As you hint at, something about this style seems particularly nice when it comes to code "evolving"
#2022-03-0500:49wilkerlucio#2022-03-0713:36wilkerlucio#2022-03-0814:35danierouxWhat am I missing here regarding parameters for pathom3?
(def eql-with-params
  ['({[:asset/id 13] [:asset.series/level]}
     {:time.series/from #inst "2021-12-11T00:00:00.000-00:00",
      :time.series/to #inst "2021-12-16T00:00:00.000-00:00"})])

(pco/defresolver series-resolver [env _]
  {:asset.series/level {:params-expected-but-is-empty (pco/params env)}})

(def env (pci/register series-resolver))

(p.eql/process env eql-with-params)

=> {[:asset/id 13] {:asset.series/level {:params-expected-but-is-empty {}}}}
#2022-03-0814:36danierouxI also explored down this path, with no clarity yet: https://clojurians.slack.com/archives/C68M60S4F/p1646737904726649?thread_ts=1646732244.136159&amp;cid=C68M60S4F#2022-03-0814:58wilkerluciohello, the problem is the position of the params, in this example the params is set at the ident level, and params don't flow down (they are only acessible at the place they were defined)#2022-03-0814:59wilkerluciothis is the query to send the params at the right place in your case:
(def eql-with-params
  ['{[:asset/id 13]
     [(:asset.series/level
        {:time.series/from #inst "2021-12-11T00:00:00.000-00:00",
         :time.series/to   #inst "2021-12-16T00:00:00.000-00:00"})]}])
#2022-03-0815:21danierouxAh. Fulcro generates it like I have it: https://clojurians.slack.com/archives/C68M60S4F/p1646732244136159?thread_ts=1646732244.136159&amp;cid=C68M60S4F Is there something obvious I'm missing?#2022-03-0821:44wilkerluciothis issue is about a conceptually mismatch in this case, for Fulcro it thinks the params are "entity level", but for Pathom the params are "per attribute", the common solution in Fulcro with Pathom 2 is to use Pathom plugin that forwards down the params from a parent entity via environment#2022-03-0821:45wilkerlucioanother thing I believe (have to test) you can do is to put the param strait in the query, instead of using the fulcro :params thing#2022-03-0821:45wilkerluciothe plugin solution doesn't exist afaik for Pathom 3 yet#2022-03-0815:15markaddlemanIn Pathom3, I use the three arity p.eql/process to keep the query separate from the entity. My "params" are folded into the entity map. I haven't encountered any problems taking this approach but I'm wondering if there is any advantage to keeping "params" separate from the entity#2022-03-0907:40yendathe docs mentions consuming graphql, but is there any plan in the future to do the other way around (propose a graphql endpoint?). my understanding is that it may be even "easier" as graphql schemas define strict entry points to the graph through queries and mutations#2022-03-0909:12jmayaalvi experimented with this a bit and would like to go back to it soon. there is an ongoing discussion here https://github.com/wilkerlucio/pathom3/discussions/18 and some beginnings here https://github.com/jmayaalv/graffito/#2022-03-0909:51aratareHi there, I'm currently using Pathom along with Fulcro and would love to get more into the REPL way of working. One thing that alludes me is how to test out mutations and resolvers against the parser. For example, let's say I have a mutation called login that takes a username and password as its args, what sort of code do I need to write where I can send it to the REPL to see the mutation code in action? Thanks in advance.#2022-03-0914:42wilkerluciothe most accurate way to test is to call the parser directly and send your mutation there, this way you make sure you running with all the plugins and etc#2022-03-0914:43wilkerlucioif you still want to call the mutation in isolation, you can grab the function via ::pc/mutate in the mutation, that key has the function with 2 arguments (env and params) to call the mutation directly#2022-03-0914:43wilkerlucioI'm assuming you are asking about Pathom 2#2022-03-0914:44wilkerlucioin Pathom 3 both mutations and resolvers are custom types that implement IFn, so you can call those directly#2022-03-0914:46aratareHey. Yes I'm using pathom2 at the moment (because Fulcro). Seems like calling the parser directly is the way I want since I want to test out the entire thing (i.e. request comes in responses goes out). Thanks a lot šŸ˜„#2022-03-1004:11nivekuilfulcro works just fine with pathom3 btw#2022-03-0917:27pithylessAnyone ever see an error like this?
1. Caused by java.lang.AbstractMethodError
   Method
   com/wsscode/pathom3/connect/operation/Mutation.applyTo(Lclojure/lang/ISeq;)Ljava/lang/Object;
   is abstract
This showed up when I had something like this:
(comment
  (def foo (my.app.mutations/some-mutation {:my :env} {:my :input})))
So, calling the mutation directly as a function and trying to bind it to a def. If I instead used (defn foo [] ...) and then (def foo2 (foo)) everything worked fine.
#2022-03-0918:42wilkerluciothat's strange, never seen that before#2022-03-1002:37Drew VerleeIn order to resolve an attribute, will pathom walk up the tree and across to a sibling tree then down it? e.g I declare an input to my resolver [:a {:c [:d]} {:b [:e]}] and :d requires :e to be resolved. Will pathom do that? does the question make sense?#2022-03-1002:56wilkerlucio:d and :e are at different levels, and you can never look up (only down), does that help?#2022-03-1003:03wilkerlucioif you get a more concrete example we can talk over it#2022-03-1003:20Drew VerleeI suppose I had been thinking of Pathom as a "graph language" but i'm realizing that might be not a good mental model. in datalog you could do a join across. In pathom assume i need to pull the information i need (at level :e) using a query inside my resolver and then inject it at the other level (:d) to get the desired effect. Yep thats exactly what were doing in other places in our codebase. Ok i understand.#2022-03-1013:37wilkerlucioit is a graph in the sense you navigate the entities, but not in the sense of datalog that you do this kind of disparate joins, Pathom doesn't optimize for query in that sense, instead of about entities and how they relate to each other, the resolvers allow the navigation, but something like a search is more suited to just be given a name, and use some underlying language (like datalog, sql, etc...) to do the find and return the entities to be further navigated by Pathom, makes sense?#2022-03-1014:51StefanHi all! We’re doing our first Pathom (3) steps in our code. We’re now considering how to do pagination. We can see two alternatives. One is this:
(pco/defresolver user-resolver
  [env _]
  {::pco/output [{:user/all [:user/id :user/name]}
                 :users/all-count]}
  (let [limit (get (pco/params env) :pagination/limit)
        offset (get (pco/params env) :pagination/offset)
        users (query-users)]
    {:users/all (take limit (drop offset users))
     :users/all-count (count users)}))
The other is this:
(pco/defresolver user-resolver
  [env _]
  {::pco/output [{:user/all
                  [:pagination/total
                   {:pagination/items [:user/id :user/name]}]}
                 :users/all-count]}
  (let [limit (get (pco/params env) :pagination/limit)
        offset (get (pco/params env) :pagination/offset)
        users (query-users)]
    {:users/all {:pagination/total (count users)
                 :pagination/items (take limit (drop offset users))}}))
We’re not sure which to choose, we lack the experience to understand the trade-offs. Can somebody shine their light on this for us? Thanks!
#2022-03-1014:55jmayaalvfor pagination we have opted for a cursor based approach instead of an offset based approach. Mostly because we want to avoid the count. This of course depends in your backend and how you load the data.#2022-03-1014:57jmayaalvbut in general the implementation is pretty similar to the second one.#2022-03-1116:04kendall.buchananYou’re welcome to look at how we’ve implemented it…#2022-03-1116:04kendall.buchanan
(pco/defresolver notifications-app
  [{sql :db/sql}
   {user-id     :user/id
    page-size   :page/size
    page-number :page/number}]
  {::pco/input  [:user/id
                 (pco/? :page/size)
                 (pco/? :page/number)]
   ::pco/output [{: [:page [:page/previous
                                             :page/current
                                             :page/next]
                                      : notification-app-attributes
                                      : notification-app-attributes]}]}
  (let [page-size (or page-size 20)
        page-number (or page-number 1)
        limit page-size
        offset (- (* limit page-number) limit)
        notifications (->> (store/get-notification-app-notifications-by-recipient-user sql user-id (inc limit) offset)
                           (map transform-notification))
        page-previous (if (= page-number 1) nil (- page-number 1))
        next-notification? (= (inc limit) (count notifications))
        page-next (when next-notification? (+ page-number 1))
        notifications (take limit notifications)            ;; adjust for over-fetching
        new-notifications (filter #(= :unread (:notification.app.status/status %)) notifications)
        prv-notifications (filter #(= :read (:notification.app.status/status %)) notifications)]
    {: {:page                       {:page/previous page-previous
                                                      :page/current  page-number
                                                      :page/next     page-next}
                         :      new-notifications
                         : prv-notifications}}))
#2022-03-1118:08wilkerlucio@U0HJA5ZQT looking at your source it seems the :page attribute is missing a map around it to place the :page/previous and others as children of them#2022-03-1118:26wilkerlucio:page [:page/previous :page/current :page/next] vs {:page [:page/previous :page/current :page/next]}#2022-03-1118:47kendall.buchananGood catch!#2022-03-1217:35wilkerlucio#2022-03-1507:49aratareHi there. What is the best way to catch and handle exceptions thrown within a mutation? E.g. I have a simple mutation that creates an entry in the database. If I feed bad input, the database will throw an exception but I can't seem to catch and process it with error-handler-plugin and ::pc/process-error, i.e. the provided error handling function is not triggered. I can, of course, handling it per mutation but I'd rather have a global handler that will take care of this for me. Thanks in advance.#2022-03-1507:50aratareHere's what I have for the parser
(p/parser {::p/env     {::p/reader                 [p/map-reader pc/reader2 pc/ident-reader pc/index-reader]
                          ::pc/mutation-join-globals [:tempids]
                          ::pc/process-error         (fn [env err]
                                                       (log/spy env)
                                                       (.printStackTrace err)
                                                       (p/error-str err))}
             ::p/mutate  pc/mutate
             ::p/plugins [(pc/connect-plugin {::pc/register resolvers})
                          p/error-handler-plugin
                          (p/post-process-parser-plugin p/elide-not-found)]})
As stated previously there is no stacktrace printed.
#2022-03-1520:32wilkerlucio@U013F1Q1R7G there is a typo, its ::p/process-error instead of ::pc/process-error#2022-03-1605:17aratareAh silly me šŸ˜…#2022-03-1606:18aratareIt's working perfectly now. Thank you šŸ™‚#2022-03-1613:03wilkerluciono worries, sorry for the sometimes confusing namespaces šŸ˜…#2022-03-1707:42aratareAll is well šŸ˜„ It's my fault for not paying enough attention šŸ™‚#2022-03-1614:05danierouxLenient mode in Pathom3, using boundary-interface** and serializing to Transit: :com.wsscode.pathom3.error/node-exception fails to serialize. Do we need to remove that from the result before sending over the wire?#2022-03-1614:05danieroux
(ns pathom-boundary-interface-errors
  (:require
    [cognitect.transit :as transit]
    [com.wsscode.pathom3.connect.operation :as pco]
    [com.wsscode.pathom3.connect.indexes :as pci]
    [com.wsscode.pathom3.interface.smart-map :as psm]
    [com.wsscode.pathom3.interface.eql :as p.eql]
    [com.wsscode.pathom3.connect.built-in.resolvers :as pbir]
    [com.wsscode.pathom3.error :as p.error])
  (:import
    [ ByteArrayOutputStream ByteArrayInputStream]))

(def indexes
  (pci/register
    [(pbir/constantly-fn-resolver ::throw-error (fn [_] (throw (ex-info "Error" {}))))]))

(def pathom
  (p.eql/boundary-interface
    (merge
      indexes
      {::p.error/lenient-mode? true})))

(defn ->transit
  [form]
  (let [out (ByteArrayOutputStream. 4096)
        writer (transit/writer out :json)]
    (transit/write writer form)
    (.toString out)))

; :com.wsscode.pathom3.error/exception cannot be serialized to Transit
; Throws:
; Not supported: class clojure.lang.ExceptionInfo
(->transit (pathom [::throw-error]))
#2022-03-1614:12wilkerluciowhat version are you using? I see something different on my end, but there is also a bug I notice#2022-03-1614:14wilkerluciobut on my end, the bug is that if you have ::p.error/lenient-mode? true the boundary interface will return a blank output (on latest pathom)#2022-03-1614:14wilkerlucioinstead of using at the env level, you can set it on the request level as well#2022-03-1614:14wilkerluciolike this:#2022-03-1614:14wilkerlucio
(pathom {:pathom/eql [::throw-error]
                      :pathom/lenient-mode? true})
#2022-03-1614:14wilkerlucioright now, sending it there works, I'm going to fix pathom in a bit#2022-03-1614:34danierouxCurrently on v2022.01.09-alpha#2022-03-1614:34danierouxSo the Right Thing to get things over the wire is to remove any :com.wsscode.pathom3.error/node-exception from the result?#2022-03-1615:17wilkerlucioyou should bump Pathom, you are using an old version#2022-03-1615:17wilkerlucioin the new versions it already does some error processing that makes it suitable to go over the wire#2022-03-1615:18wilkerlucioI'm finishing the fix to the latest, should be out today#2022-03-1615:19wilkerluciocurrent latest is 2022.03.07-alpha#2022-03-1708:00danierouxThank you @U066U8JQJ! We have errors flowing over the wire#2022-03-1800:02wilkerlucio@danie just released 2022.03.17-alpha which fixed the lenient mode from env with boundary interface#2022-03-1817:53danierouxWorks like a charm, thank you @U066U8JQJ#2022-03-1800:02wilkerlucio@danie just released 2022.03.17-alpha which fixed the lenient mode from env with boundary interface#2022-03-1817:10λustin f(n)In pathom2, is there a way to have a ::pc/input that requires a specific nested output? For example, if I have:
::pc/input #{::story/name}
::pc/output [{::story/current [::story/id]}]
and
::pc/input #{::story/id}
   ::pc/output [::story/versioned-story-data]
From earlier resolvers, I want to do something like...
::pc/input #{{::story/current [::story/versioned-story-data]}}
 ::pc/output [::story-ana/current-reporter]
Is this possible in pathom2, if so what is the syntax?
#2022-03-1817:53λustin f(n)Ah. Given how much attention is given to Nested Inputs in Pathom3, this is probably a new feature I can't use in 2#2022-03-1817:53λustin f(n)https://pathom3.wsscode.com/docs/resolvers/#nested-inputs#2022-03-1817:54wilkerlucioyes, that's correct, this is a feature only supported in Pathom 3#2022-03-2116:00Reily SiegelI am looking into the feasibility of making a general purpose resolver generator for HTTP APIs in pathom3. I have come up with a very early stage example that can take a configuration and generate resolvers for each endpoint. The idea here is that each endpoint specifies the pathom attributes it needs for the header, query, body, and path parameters, and also what attributes the resolver outputs. It then generates a resolver that calls the API, and possibly transforms the result. Presumably this configuration could be automatically generated from API specifications like OpenAPI or Swagger, vastly simplifying the usage of pathom with external data sources. I have attached a sample configuration as a reply (so as not to clutter the channel). Do you think something like this is worth persuing?#2022-03-2116:00Reily Siegel
(resolvers
     { ;; Base URL for the API
      ::http/url ""
      ;; Function that processes an HTTP request, and returns a Clojure map with
      ;; strings as keys. Must accept :url, :query, :headers, :body, :method
      ::http/request!
      (fn [req]
        (j/read-value (:body @(client/request req))))
      ;; Specify the endpoints of the API
      ::http/endpoints
      [{;; The method of the endpoint
        ::http/method :get
        ;; Attributes that should be added to the path, in order. Items that are
        ;; not keywords are assumed to be constants.
        ::http/path   ["people" :person/id]
        ;; Attributes the resolver will output
        ;;
        ;; By default, pathom-http expects the API to produce a
        ;; string key of the name part of each keyword. For
        ;; example, for the keyword :person/name, pathom-http
        ;; will look for the string key "name" in the response.
        ::pco/output  [:person/name :person/height  :person/mass
                       :person/hair_color :person/eye_color
                       :person/gender :person/birth_year
                       :person/homeworld :person/films
                       :person/species :person/vehicles
                       :person/starships
                       :person/url]}
       {::http/method :get
        ::http/path   ["planets" :planet/id]
        ::pco/output  [:planet/name :planet/rotation_period
                       :planet/orbital_period :planet/diameter
                       :planet/climate :planet/gravity
                       :planet/terrain :planet/surface_water
                       :planet/population :planet/residents
                       :planet/films :planet/url]}]
      ;; Transformations that should be applied to attributes
      ;;
      ;; Here, SWAPI returns URLs to related attributes, rather than IDs. The
      ;; last-segment transform gets the last segment of a URL, and returns it
      ;; as the value of the attribute. The last-segments transform, similarly,
      ;; does the same thing, but works on an array of URLs, and requires an
      ;; attribute to associate with each ID. Transforms happen after
      ;; translations.
      ::http/transform
      {:person/homeworld last-segment
       :person/films     (last-segments :film/id)
       :person/species   (last-segments :species/id)
       :person/vehicles  (last-segments :vehicle/id)
       :person/starships (last-segments :starship/id)
       :planet/residents (last-segments :person/id)}})
#2022-03-2214:18wilkerlucioI think its great, usually the problem is around parameters, which sometimes are hard to figure which attributes they should map to. this is also a problem in GraphQL integration, that's why there is the manual mapping of it there#2022-03-2200:03sheluchinI'm trying to run this GitHuh GraphQL query with Pathom but it's not working as I expect.
{
  search(query: "sheluchin", type: USER, first: 1) {
    edges {
      node {
        ... on User {
          login
        }
      }
    }
  }
}
In Pathom:
[{(:github.Query/search {:query "sheluchin" :type USER :first 1})
  [:github.SearchResultItemConnection/userCount
   {:github.SearchResultItemConnection/edges
    [{:github.SearchResultItemEdge/node
      [:github.User/login]}]}]}]
Result:
cause "Pathom can't find a path for the following elements in the query: [:github.User/login] at path [:github.Query/search :github.SearchResultItemConnection/edges :github.SearchResultItemEdge/node]"
Might this be a bug?
#2022-03-2214:31wilkerlucioit looks similar to https://github.com/wilkerlucio/pathom3-graphql/issues/15#issuecomment-1069050373 to me, I plan to do the debugging on this by Thursday this week, hopefully its same thing and we can get both fixes together šŸ¤ž#2022-03-2214:53sheluchinOk, @U066U8JQJ. Thanks for checking it out. Hopefully it is related.#2022-03-2211:47Mark WardleHi all. I now have 153 resolvers and counting, some in my main server product, and many others as separate resolvers in specific libraries. I use integrant to configure much of the environment, so that resolvers have access to different connections to databases, key value stores etc.. But I don't have an easy way of registering resolvers as part of that process. I currently add the resolvers in my integrant init method, storing them in an atom, and finally making use of the collection when the pathom environment is created. e.g. see https://github.com/wardle/pc4/blob/49c3b50667c92d0c35e244de8ba8566888786fd4/pc4-server/src/com/eldrix/pc4/server/system.clj#L103. This works well at small scale, but I'd like to move to being able to keep resolvers and the service init/halt multimethods in their own namespaces. Here is my example integrant configuration file: https://github.com/wardle/pc4/blob/main/pc4-server/resources/config.edn Has anyone got any suggestions on how to use pathom with integrant to register the appropriate resolvers if and when a specific service is configured? Thank you, Mark#2022-03-2214:27wilkerluciohello Mark, I was playing with integrant setup, I have a half-baked Pathom 3 integrant library, but its development its currently on pause#2022-03-2214:27wilkerlucioin my setup, the config looks like this:#2022-03-2214:27wilkerlucio
{:com.wsscode.pathom3.integrant/pathom3
 {:com.wsscode.pathom3.integrant/registry
  [#pathom3/ops some.org.sample-resolvers/registry
   #pathom3/ops "some\\.org\\..+"]

  :com.wsscode.pathom3.integrant/env
  {}

  :com.wsscode.pathom3.integrant/pathom-viz-enabled?
  false

  :com.wsscode.pathom3.integrant/pathom-viz-name
  "demo"}}
#2022-03-2214:27wilkerlucioso you can reference resolvers (by var or regex namespace match) to be included#2022-03-2214:27wilkerluciothis is the current source, not much but may help you as a starting point:#2022-03-2214:27wilkerlucio
(ns com.wsscode.pathom3.integrant
  (:require [integrant.core :as ig]
            [clojure.tools.namespace.dir]
            [com.wsscode.pathom3.connect.operation :as pco]
            [clojure.spec.alpha :as s]
            [some.org.sample-resolvers]
            [ :as io]
            [com.wsscode.pathom3.connect.indexes :as pci])
  (:import (java.util.regex Pattern)))

(s/def ::registry ::pci/operations)

(defn re-find-operations [re]
  (into []
        (comp
          (filter #(re-find re (str %)))
          (map ns-publics)
          (mapcat vals)
          (map var-get)
          (filter pco/operation?))
        (all-ns)))

(defn find-operations [x]
  (cond
    (qualified-symbol? x)
    (var-get (requiring-resolve x))

    (string? x)
    (re-find-operations (re-pattern x))))

(def readers
  {'pathom3/ops find-operations})

(defmethod ig/init-key ::pathom3 [_ {::keys [registry] :as p}]
  (let [env {}]
    (pci/register env registry)))

(comment
  (var-get (requiring-resolve 'some.org.sample-resolvers/registry))

  (ig/read-string
    {:readers readers}
    (slurp (io/resource "sample_app/config.edn")))

  (let [publics (ns-publics ns)])
  (->> (find-ns 'some.org.sample-resolvers)
       (ns-publics)
       vals
       (mapv #(var-get %)))
  (find-operations #"^some\.org\..+-resolvers$")
  (clojure.tools.namespace.dir/dirs-on-classpath))
#2022-03-2214:29wilkerluciofor the symbol version this code can load the namespace dinamically, but note that for the namespace regex version you still responsible for loading the possible namespaces yourself, then the code can match from those loaded namespaces#2022-03-2214:50Mark WardleOooo that's clever. What a nice idea. I haven't used clojure's ability to find public symbols in namespaces before, but that would work really well. I shall have a good play with it! Thank you. Hope you are keeping well. Mark#2022-03-2215:22wilkerlucioglad you liked it šŸ™‚ IMO the option using explicit vars (pointing to registry entries) is preferred over the regex, being explicity makes easier to track and understand whats going on#2022-03-2415:55Bjƶrn EbbinghausHas anyone here experimented with Pathom3 as a client-side pull-trough cache as a fulcro remote? Or generalized: With the ability to connect pathom3 parsers, it could enable behaviour like a Datomic peer. A local parser, that resolves as much as it can from a local store and queries for the missing parts from the actual remote parser.#2022-03-2522:19wilkerluciohave not done any experiments on this direction yet, if you do please let me know how it goes šŸ™‚#2022-03-2509:42geraldodevconsider ::pco/output of the department->manager , could you explain what are the possible differences/implications of them. all three works
(pco/defresolver department-id->department [{:keys [db-hr]} {:keys [department/id]}]
  {::pco/input [:department/id]
   ::pco/output [:department/id
                 :department/department_name
                 :department/manager_id
                 :department/location_id]}
  (namespace-flat-map
    "department"
    (jdbc/execute-one!
      db-hr
      ["select department_id, department_name, manager_id, location_id from hr.departments where department_id=?" id]
      {:builder-fn rs/as-unqualified-lower-maps})))

(pco/defresolver department->location [{:keys [department/location_id]}]
  {::pco/output [:department/location]}
  {:department/location {:location/id location_id}})

(pco/defresolver department->manager [{:keys [department/manager_id]}]
  {::pco/output [{:department/manager [:employee/id]}]}
  {:department/manager {:employee/id manager_id}})

; (pco/defresolver department->manager [{:keys [department/manager_id]}]
;   {::pco/output [:department/manager]}
;   {:department/manager {:employee/id manager_id}})

; (pco/defresolver department->manager [{:keys [department/manager_id]}]
;   {::pco/output [{:department/manager [:foo]}]}
;   {:department/manager {:employee/id manager_id}})
#2022-03-2515:07Bjƶrn EbbinghausSo: You resolve :employee/id coincidental through :department/manager meaning, that if you have some other resolver that provides :department/manager but not :employee/id it can be possible that pathom won't be able to resolve :employee/id Imagine you add another resolver:
(pco/defresolver something->manager [{:keys [something]}]
  {:department/manager something})
And you run the query:
[{:department/manager [:employee/id]}]
How should Pathom decide which resolver to use? Maybe it chooses something->manager to resolve :department/manager and has no clue how to get to :employee/id . Additionally, you lose things like autocomplete and documentation in Pathom Viz. From https://pathom3.wsscode.com/docs/resolvers#using-defresolver`defresolver` > IMPORTANT > Both input and output can express flat or nested queries, you should always strive to be as precise with these descriptions as you can (matching the shapes of your inputs and outputs), Pathom leverages this information in many ways during transaction processing, and as more accurate you get, the more Pathom can do with it. You can return more data than you defined, but you should only do that "as a bonus" and you shouldn't rely on that.
#2022-03-2515:36geraldodevThank you @U4VT24ZM3!#2022-03-2516:07Bjƶrn EbbinghausAnd: You can elide the ::pco/output config in this example, as you directly return just maps.#2022-03-2715:29geraldodev
(pco/defresolver employee->manager [{:keys [employee/manager_id]}]
  {::pco/input [:employee/manager_id]
   ::pco/output [{(pco/? :employee/manager) [:employee/employee_id]}]}
  {:employee/manager (when manager_id {:employee/employee_id manager_id})})
Today I've learnt two things : One is that the optional atributte (pco/? attr) must be defined at attribute level. I was mistakenly calling the entire output shape with (pco/? {...}). The second thing and that is not intuitive for me is I need to return the {:employe/manager nil} in case of the attribute being nil. I was mistakenly returning nil instead of the map with the nil value and it was getting "Required attributes missing: [:employee/manager] " I wonder required by whom , the calling query ? Because I've already marked it as optional. I also learnt that I can mark the attribute as optional at the calling query, but I don't think it's a good solution.
#2022-03-2715:46wilkerluciohello Geraldo, marking output things as optionals have no meaning, the requirement of something is always on the loader side (the input), outputs are are always "optional by default", the requirement is always on the demanding side (which may flag it as optional)#2022-03-2715:47wilkerlucioso when it says Required attributes missing: [:employee/manager] it means some part of your query requested that as required (which is the default, unless you mark something as optional, that is true both for resolver inputs, as for your actual query, which you can also mark things as optional if that's the case)#2022-03-2715:47wilkerluciothat makes sense?#2022-03-2719:38geraldodev
;; no manager
  (p.eql/process
    (pci/register (user/db-hr-map) index)
    [{[:employee/employee_id 100] [:employee/employee_id :employee/first_name :employee/salary
                                   {:employee/manager [:employee/first_name]}]}])
I've read again the optional inputs documentation with the context of your commentary and it really makes the case of inputs like your are teaching me. It's very hard to put on words how an algorithm is working. :employee/manager and employee/first_name are output properties and both are declared on the query, but for :employee/manager I have to return an attribute with a null value and when that is the case there is no :employee/first_name and no message of it being required. What I'm experiencing is output properties are optional and as long the engine does not turn the attention for it in a resolver.
#2022-03-2719:55geraldodevThank you for teaching us.#2022-03-2823:40geraldodevpprinting env with its forest of juicy clojure data structures challenge my untrained eyes, with sets and maps together šŸ™‚#2022-03-2901:50wilkerlucioI suggest using #portal or #reveal to assist in navigating large data structures like these :)#2022-03-2906:05HukkaIndeed, I always run
clj -A:dev -Sdeps '{:deps {nrepl/nrepl {:mvn/version "0.7.0"} cider/cider-nrepl {:mvn/version "0.25.2"} djblue/portal {:mvn/version "0.20.1"}}}' \
           -m nrepl.cmdline \
           --middleware '["cider.nrepl/cider-middleware"]' \
           --interactive
so that I can start tapping into portal when needed without restarting the repl and losing state, even if I don't need it more than couple times in a month. With this spell in the shell history I can have the tools ready in every project
#2022-03-3010:43geraldodevThnx for the suggestion I'll try #portal#2022-03-3020:08markaddlemanIn my Pathom 3 app, I have a resolver that reads and interprets an edn data file. Within that data file are pathom queries that should be resolved under the context of the loading resolver as part of its interpret operation. So, I reuse the resolver's env when executing the data file's pathom queries. Are there any performance or functional implications of reusing the env this way? The only (very minor) side affect of this approach is that the Pathom Visualizer sees the data file queries as separate queries.#2022-03-3020:22wilkerlucioits fine to use env in this way, one thing to keep in mind is that the env from inside a resolver will already contains some caches (both standard and internal ones) that you may want to elide when doing the next call, you can use the helper (pcp/reset-env env) to get a hold of a clean version (in terms of caches and processing things) of the env#2022-03-3020:22wilkerlucioso the call may look like: (p.eql/process (pcp/reset-env env) [...])#2022-04-0219:21nivekuilcan I get ::pcr/node-resolver-input within ::pcr/wrap-resolver-error? I want to log the params being passed to the resolver that had an error but it's always nil in the plugin#2022-04-0414:01wilkerluciohello, I have to try myself to understand this better, will do it later today#2022-04-0412:34nivekuilI'm looking at a node I expect to be green in pathom-viz but it's grey, and the error is this. relevant or just cosmetic?#2022-04-0412:51nivekuilI think that's just cosmetic since every grey node has it.. I'm puzzled over this though:#2022-04-0412:51nivekuilthis OR node says the success-path is 71, which is the AND node#2022-04-0412:52nivekuilbut entry has already provided enough info#2022-04-0412:52nivekuilwhat the OR node wants is embed/html which is provided already as nil#2022-04-0412:52nivekuilso it seems like it's still walking extraneous resolvers? hard to repro this one though#2022-04-0414:01wilkerluciointeresting one, I think I'm not checking for results ready in the OR node, so it does pick a path anyway, but this should be an easy fix, can you open an issue for that please?#2022-04-0419:56wilkerlucioI just checked the code, the OR node does check for completion already, so that shouldn't be an issue, the grey means it got there but there was no need to run, so its skipped#2022-04-0419:57wilkerlucioabout the pluzzing error message, that's a bug in pathom viz, that error was not related to your project, but was Pathom trying to calculate the time spent there (which doesn't exists, given the node didn't ran)#2022-04-0419:57wilkerlucioIm missing something?#2022-04-0419:58wilkerlucioalso, the order of execution might matter here, maybe it entered this OR node before the entry (the green one)#2022-04-0419:58wilkerluciogrey nodes also happen for paths not taken in a OR branch#2022-04-0515:08nivekuilwell since entry provides a superset of the data that the OR node needs I wouldn't have expected to it be run in that order anyway#2022-04-0515:09nivekuilalso it's higher priority. I can check, I have the graph data saved but I can't find the tool to import graph data on the pathom3 website#2022-04-0515:10nivekuilthis was run with parallel, maybe that could affect order of execution?#2022-04-0517:36wilkerlucioyes, parallel can certainly affect the order of execution#2022-04-0517:36wilkerluciolooking at the timeline you may be able to see what ran first#2022-04-0517:38wilkerluciohumm, but there is a problem, I double checked the code, the check is only happening on the serial runner, I'll add the same check for async and parallel today#2022-04-0601:54wilkerluciomissed check on parallel and async are now fixed in main#2022-04-0614:22nivekuilseems like you got it šŸ™‚#2022-04-0413:53StefanHi all, we’re running into a bit of a performance problem with Pathom 3 that I’d like to understand better. I hope someone can help. Please consider the following resolver:
(pco/defresolver all-organisations [_]
    {::pco/output [{:organisation/all [:organisation/id
                                       :organisation/name
                                       {:organisation/departments [:department/id]}]}]}
    {:organisation/all
     (map (fn [id] {:organisation/id id
                    :organisation/name (str "org-" id)
                    :organisation/departments [{:department/id 1
                                                :department/name "department-1"}]})
          (range 1 10))})
This is a resolver that normally goes out to the database, but for the sake of this discussion I’m just generating some example organisations. Now I do this query:
[{:organisation/all [:organisation/id
                     :organisation/name
                     {:organisation/departments
                      [:department/id :department/name]}]}]
The results are OK, but when I do this for a thousand organisations, it takes maybe five seconds or so. Looking in Pathom Viz I see what I captured in the attached screen shot. As you can see, a lot of planning is going on for each department in each organisation, whereas all data is immediately available. Is this because I’m doing something wrong? Is this maybe a missing optimisation step in Pathom? Thanks!!
#2022-04-0413:53StefanBy the way, this is not the same issue as https://github.com/wilkerlucio/pathom3/issues/134, I checked :)#2022-04-0413:59wilkerluciohello Stefan, one thing that you can improve here may be to use a vector mapv instead of map, Pathom handles vectors better in most cases and that should be preferred. about the sequence processing, Pathom has to plan for every entry, but this should hit a cache hit after the first, how many items are we talking about? can you make a repro? I have some ideas to improve even further the planning on sequences, but was waiting for some real case to put it forward, maybe this is the one, but like to check the possible edges first#2022-04-0415:30pithylessJust out of curiosity, would the planner work better if you split the joins into separate resolvers and used the batch resolver? (It may also change the I/O contention and GC churn in a positive manner, i.e. theoretically more work but overall lower latency)?#2022-04-0416:48wilkerluciobatch doesn't participate on planning in any way, its an operation during the running process only. when the runner detects a batch item, it will take note of that demand, pause the current entity execution and jump to the next entity, once the whole query is scanned, them it will look at every batch demand annotated, invoke them, merge results and then resume the process#2022-04-0416:49wilkerlucioits a bit different on parallel, because in parallel there is no clear "scan ending" due to to many things being happening at the same time, so for batch there it uses a time window to group items (default 10ms if I remember right), accumulate and run#2022-04-0421:04Stefan@U066U8JQJ Thanks for your suggestion. Using mapv doesn’t seem to make a noticeable difference in this case unfortunately. I think this is the complete reproduction, expanding on the code above:
(require '[com.wsscode.pathom3.interface.eql :as p.eql])
  (require '[com.wsscode.pathom3.connect.operation :as pco])
  (require '[com.wsscode.pathom.viz.ws-connector.core :as pvc])
  (require '[com.wsscode.pathom.viz.ws-connector.pathom3 :as p.connector])

  (pco/defresolver all-organisations [_]
    {::pco/output [{:organisation/all [:organisation/id
                                       :organisation/name
                                       {:organisation/departments [:department/id]}]}]}
    {:organisation/all
     (mapv (fn [id] {:organisation/id id
                     :organisation/name (str "org-" id)
                     :organisation/departments [{:department/id 1
                                                 :department/name "department-1"}]})
           (range 1 1400))})


  (p.eql/process (-> (pci/register [all-organisations])
                     (p.connector/connect-env {::pvc/parser-id ::env}))
                 {}
                 [{:organisation/all [:organisation/id
                                      :organisation/name
                                      {:organisation/departments
                                       [:department/id :department/name]}]}])
#2022-04-0421:05StefanThis takes about 3 seconds on my machine. And 1400 entities is not that big a number of course…#2022-04-0421:09wilkerluciothanks for the repro, I'll take a closer look over this week#2022-04-1110:19Stefan@U066U8JQJ Did you get around to looking into this? Is there anything I can do to help you? (Besides learning the internals of Pathom for which I’m afraid I don’t have enough time right now šŸ˜…)#2022-04-1116:58wilkerluciohello @UGNFXV1FA, sorry the delay, I recently got a pet and now a lot of my time have been dedicated to take care of her, rsrs. I did some checks, and the short answer is that this is expected overhead for Pathom. Im starting to write a long answer for this, because I think its good to have a well explained text on the performance characteristics of Pathom. for now, I can tell you if the focus is on raw processing performance, Pathom does add a considerable overhead when processing 1000s of records (about 0.5ms per entity). I saw raw processing because that's when we consider CPU time. But a lot of Pathom usage usually has the bottleneck on IO, and in those cases, using the parallel processor feels more like a garbage collector, something that you could in simple cases make it more performant manually, but when doing complex processes Pathom can optimize things in a way that would be unfeasible to do manually (and maintain), and this makes the Pathom overhead negligible. as I said, I'm going to prepare a long answer to explain why is that in more detail (why the overhead). at this point I think the only way to improve is on micro improvements (I even tried clj-fast, but for some reason it got slower when I did...), or changing the language or the data structures, I'm open to suggestions but at this point I don't see much to done in the short term#2022-04-1116:59wilkerlucioI'm still going to play with the plan for sequences, but that is problematic too, because Pathom focus on dynamism and correctness, and Pathom doesn't expect a sequence of items to have regular data (which means we need a plan for each entity to ensure correctness)#2022-04-1119:38StefanThanks for your clarifications @U066U8JQJ. For my case, focus is not necessarily ā€œraw performanceā€, but ā€œgood enough performanceā€ šŸ˜‰ The data that we’re querying (e.g. ± 1000 organisation names) is not that much, and when this processing time is 3–5 seconds it is a bit off compared to the I/O. In other words, I/O is definitely not the bottleneck anymore. Of course I completely agree with the goal of correctness, and the dynamism, but I’d expect this to be a quite common case. I don’t understand enough of the details to make good suggestions I’m afraid, but is it worth thinking about some kind of hint (metadata) that we could pass to Pathom to allow it to make certain assumptions and be more efficient?#2022-04-1119:44wilkerlucioyeah, that's something I'm down to try for sure (specific meta to improve)#2022-04-1119:44wilkerluciomost of the time, since we are doing UI's render, the UI is paginated, so we can go fine loading like 50 items of each thing at time#2022-04-1119:45wilkerluciobut for times where we need to work with larger lists, there is a scape hatch, you can mark something as final, and by doing so Pathom wont perform any sub-query parts (so its up to you to ensure all the data is there)#2022-04-1119:46wilkerlucioso what you suggesting I see more in the middle terms, like trying to mark that a list is consistent, and have the plan doing only once for every item#2022-04-1119:47wilkerluciowe have to test, I expect this optimization may get about 20% ~ 40% reduction in overhead, but there is still a lot Pathom has to do (even when there is nothing to do, Pathom has to figure that there is nothing to do, and that has some cost)#2022-04-1119:52StefanYeah that makes sense. Let me know if/when you would like me to test something on our production code. The use case by the way is that we have a (paginated) table of data that contains an ā€œorganisationā€ column. This column has a filter (select box), and the values of that filter need to be populated by all possible organisation names.#2022-04-1119:53wilkerlucioyeah, for that simple collection case (just name and id I guess) its easy to leverage the final, let me give you an example here (I just notice its not in the docs, just went on the queue for that ;))#2022-04-1119:53wilkerlucio
(pco/defresolver all-organisations [_]
  {::pco/output [{:organisation/all [:organisation/id
                                     :organisation/name
                                     {:organisation/departments [:department/id]}]}]}
  {:organisation/all
   (with-meta
    (mapv (fn [id]
           {:organisation/id          id
            :organisation/name        (str "org-" id)
            :organisation/departments [{:department/id   id
                                        :department/name "department-1"}]})
      (range 1 1400))
     {::pco/final true})})
#2022-04-1119:54wilkerlucionote you can't mark the root map as final, you have mark things inside of it (like in this example we maked the :organisation/all as final)#2022-04-1120:24StefanI’m going to try this first thing tomorrow, thanks Wilker!#2022-04-1408:54Stefan@U066U8JQJ Using final makes a huge difference.#2022-04-1408:55StefanWhat would be the ā€œidiomaticā€ way of using it? Because it depends on the use case whether it should be treated as final or not. Should it be a query parameter that decides it?#2022-04-1411:42wilkerlucioI suggest using a different attribute name for the final version, maybe something like :organization/all-for-menu-final, so when you refer to this attribute you know the constraints#2022-04-1412:57StefanThat’s a good suggestion, thanks!#2022-04-0818:56sheluchinIf you have Items and Sales of those individual items, how would you model something like "total items sold in the past month" using attribute modelling? I see a few alternatives to doing it: • don't mention anything about the date range: {:item/id 1 :item/sales-count 42} • mention the implied range past-month in the context of the Item: {:item/id 1 :item/total-sold-past-month 42} • mention the implied range in the context of Sales: {:sale/item 1 :sale/past-month-count 42} • combine contexts and be explicit about the beginning of the range with since: {:item/id 1 :sale/total-count 42 :sale/since 2022-03-08} • combine contexts and be explicit about both ends of the range: {:item/id 1 :sale/total-count 42 :sale/since 2022-03-08 :sale/until 2022-04-09} • exclude the :item/id in the output since it's in the input: {:sale/total-count 42 :sale/since 2022-03-08 :sale/until 2022-04-09} • could also be more like the first one like but with no :item/id: {:sale/total-count 42} And I'm sure there are many other conceivable ways to model it. My thinking is that it would be best not to hide any information, such as the actual date range, by being explicit by both the beginning and the end of it. This would allow the requirements may grow in the future as I could end up needing counts from the past day, week, year, or any arbitrary date range. I'm having a bit of a hard time arriving at a clear model, particularly when spanning multiple relations and filters + aggregations are involved. Can anyone offer some advice?#2022-04-0819:20wilkerluciothis is a really open place I think, and the answer will depend on the variations your system needs. as a rule of thumb I suggest starting with more specific names that fulfill some specific purpose of the application, but if you find yourself having to create too many of those, then its a good time to think about using more explicit params, that could be in the form of inputs (as you demonstrated) or as params#2022-04-0819:27sheluchinIs there much value in including all of the relevant information in the response? Like, if you're using params for a date range, does it make sense to reflect the parameter data in the response? As a novice with Pathom, it sounds like a good idea, but maybe it has limited practical value. I'm getting a bit of analysis paralysis with all the options and understanding what is helpful in the longer term. In the short term I've made a bunch of specific names and my system does what I need, but it feels like I have an explosion of names and it will not scale well with requirements. I'm at the revision stage here.#2022-04-0819:45Bjƶrn Ebbinghaus:item/total-sold-past-month is very explicit. (As long as you don't hold on to that data for long) Do you want to display it in your UI? Then that attribute is enough. You can add other names later. It is not a breaking change. === Calling it :sale/total-count would be a lie, wouldn't it? It is not the total count. It is a count in a range. {:item/id 1 :item/sales-in-range #:sale-range{:no-of-sales 42, :since ..., :until ...}} If someone is reading your code and finds :sale-range/no-of-sales they instantly knows that this is a number of sales in a given range, no need of explicit documentation. Else you have to write documentation like this: > :sale/total-count denotes the total number of sales of something, EXCEPT when it is in a map together with the keys :sale/since. === Being explicit with names allows you to trust your names and that helps a lot with preventing bugs. Example: Instead of: {:user/email " use: {:user/validated-email " An unvalidated email is not the same as a validated email, is it not? If you write a function like this:
(defn reset-password [user]
  (send-new-password (:user/email user)))
You have to think: "Who is responsible for enforcing a rule like "only send new passwords to validated emails"?" "Should I do that? Maybe the caller already validated that?" If you write instead:
(defn reset-password [user]
  (send-new-password (:user/validated-email user)))
You can trust the name. You expect a validated mail. It is the callers job to provide that. Your job is to send that mail! The caller only provided a {:user/email " ? You blow up instead of violating an important security rule.
#2022-04-0819:58Bjƶrn EbbinghausThe great thing with Graph APIs like pathom is, that you can provide as much data as you like. It won't impact anything, as long as you aren't explicitly asking for it.#2022-04-0911:12sheluchinThank you for such a detailed response @U4VT24ZM3. Your post has given me much direction and still more to think about. Taking your advice into account, the main things I'm wondering about now are: • avoiding nesting: how much is pragmatic? • introducing namespaces (:sale-range) without matching files: is this okay or should I maintain that each ns has a file? • adding dimensions (:sales-person):
{:item/id 1
 ;; Alternate ways of identifying sales by sales-person:

 ;; also adding alias from :item/sales-person -> :sales-person/id
 :item/sales-person 100
 ;; or just embedding the subtree
 :item/sales-person #:sales-person{:id 100}
 :item/sales-in-range #:sale-range{:no-of-sales 42
                                   :since <date>
                                   :until <date>
                                   ;; same representation options here as above
                                   :sales-person 100}}
 
The embedding the sales-person reference seems to lead to the same documentation requirement as you noted above. I'd have to add documentation like this: > :item/sales-in-range denotes total number of sales of something, EXCEPT when it is in a map together with the keys :item/sales-person And the alternative seems to be instead using :item/sales-in-range-by-salesman, but this is getting quite verbose and I'm not sure if embedding the entire search criteria in the name like this is practical. Perhaps replacing :item/sales-in-range #:sale-range{} with :item/filtered-sales #:filtered-sales{} would be the best alternative here? Then the search criteria could grow due more easily due to its generic parent.
#2022-04-0913:11Bjƶrn Ebbinghaus@UPWHQK562 I wouldn't provide:
{:item/id 1,
 :item/sales-person 100}
By doing this, you essentially merge the two entities. For example, if you are using a Fulcro client, this would make it impossible to normalize:
: not normalized
{:all-sales-persons {:sales-person/id 100, :sales-person/name "Bjƶrn"}
 :sold-items {:item/id 1, :item/sales-person 100, :sales-person/name "Bjƶrn"}}

; would normalize to:
{:sales-person/id {100 {:sales-person/id 100, :sales-person/name "Bjƶrn"}}
 :item/id {1 {:item/id 1, :sales-person/name "Bjƶrn"}} ;; <-- You have a duplicate!
Which is inconvenient if, e.g., you want to change the name of the sales-person.
; Better:
{:sales-person/id {100 {:sales-person/id 100, :sales-person/name "Bjƶrn"}} ;; <-- Single place where it is recorded that the name of the sales person with id=100 is "Bjƶrn"
 :item/id {1 {:item/id 1, :item/seller [:sales-person/id 100]}} 
You can of course provide a shortcut, when the client doesn't have these requirements. {:item/id 1, :item.seller/name "Bjƶrn"} Nested inputs in pathom3 are very useful for these shortcuts:
(defresolver item->seller-name [{:keys [item/sale-person]}]
  {::pco/input [{:item/sale-person [:sales-person/name]}]}
  {:item.seller/name (:sales-person/name sale-person)})
But having said all that. In the end, it's really at your discretion. What are your requirements?
; Maybe overkill? Do we really need to consider the currency as a seperate entity? 
{:item/id 42, :item/price {:price/value 100, :price/currency {:currency/id :currency/dollar, :currency/symbol "$"}}}

; Maybe this is enough?
{:item/id 42, :item/price {:price/value 100, :currency/symbol "$"}

; Or even:
{:item/id 42, :item/price-tag-dollar "$100"}
#2022-04-0913:29Bjƶrn EbbinghausRegarding your long names: When these things grow, the names will get less precise. It is good to avoid it, but you can't always do it. Are you really in the scope of an item any more? Maybe something like this is more suitable for you:
#:sale-statistic{:item #:item{:id 1, :name ā€žCar"}
                 :seller #:sales-person{:id 100, :name ā€žBjƶrn"}
                 :volume 42
                 :since #inst 2020, 
                 :until #inst 2021}
Statistic report for sales by Item:
{:item/id 1
 :item/name "Car"
 {:item/sell-statistics 
  [#:sale-statistic{:seller {:sales-person/id 100}
                    :volume 42
                    :since #inst 2020, 
                    :until #inst 2021}]}}
Statistic report for sales by seller:
{:sales-person/id 100
 :sales-person/name "Bjƶrn"
 {:sales-person/sell-statistics
  [#:sale-statistic{:item {:item/id 1}
                    :volume 42
                    :since #inst 2020, 
                    :until #inst 2021}]}}
And many more:
#2022-04-0913:37Bjƶrn EbbinghausTo summarize: Names are hard.#2022-04-0915:32sheluchin@U4VT24ZM3 that is extremely valuable advice. Thanks for taking the time to think it through and give your input here. I think you've provided enough of a framework for me to use throughout this iteration of my application. Should I miss something or arrive at something sub-optimal, it can get addressed in the next round. This round will advance my thinking nonetheless. By the way, are there any resources that can help learn this approach, or is it basically just accumulated through experience? I've gone through the Pathom and Fulcro tutorials, and those definitely helped... I just think that more examples with explanations thereof will help re-contextualize the practice in more ways and go a long way toward cementing the habit. I search around on GitHub for example usages often enough, but without the added explanation, doing all the code interpretation and trying to fit it into a mental framework of best practices is kind of hard. > To summarize: Names are hard. Yep, sure are. I feel like Pathom brings the hard part of programming - naming things - to the surface and makes you confront the difficult part early in the process. Mostly off-topic now, but the https://leanpub.com/elementsofclojure/read_sample book has a whole chapter on naming things, and mentions natural vs. synthetic names, where synthetic names are totally random have no intuitive mapping to the thing being named. I wonder if anyone has tried using synthetic names with Pathom šŸ˜„#2022-04-0917:33Bjƶrn EbbinghausRecently I got some knowledge from "Domain Modelling Made Functional" by Scott Wlaschin. Here is a talk about it from him: https://www.youtube.com/watch?v=2JB1_e5wZmU He is using F#, which is a strongly typed language, but the ideas translate to Clojure as well. You can use names and specs instead of types. In the video, he talks about getting rid of flags (like :user/email-verified?) at the 39 minute. He does it by wrapping the type EmailAddress with a Type VerifiedEmailAddress. In Clojure, you would do something similar with names and specs.
(s/def :user/email (s/and string? #(contains? % \@))) ; Add some Regex here
(s/def :user/verified-email :user/email)
#2022-04-1120:03sheluchin@U4VT24ZM3 Thanks for the video. Pretty good talk indeed. I think there is some limitation to the "just add another type" approach. From Email -> VerifiedEmail it's pretty straight forward, but sometimes entities have a whole bag of flags and it seems like this approach won't get you far in a case like that. You'll just end up with a bunch of flags encoded as separate names and I don't know if that adds much clarity in the long run. Makes me want to check out more DDD content though.#2022-04-1211:36Bjƶrn EbbinghausThat's an argument against static typing. You can't possibly name every type. @U066U8JQJ talks about that in his talk https://www.youtube.com/watch?v=YaHiff2vZ_o
{:user/id 42} ; "IdOnlyUser"
{:user/id 42, :user/email "
Madness! But since we don't have explicit types, we can just look at the keys of an entity, to determine what it is. > Sometimes entities have a whole bag of flags Can you give an example? Are there really situations, where you are better off giving things a lot of flags instead of other attributes? Flags don't really lessen the problem of naming, do they? Instead of type names, you now have flags. Even more: Now your attribute names don't have meaning on their own. They have different meaning based on the value of another name!
#2022-04-1100:47Joe R. SmithHello! I could use a little direction figuring out how to get a "boundary-interface"-defined pathom endpoint to support introspection and tracing with Pathom-Viz. The only docs I can find for pathom+pathom-viz suggest using the pathom-viz-connector function connect-env, which starts a webserver. I want to use it against a route defined in my Pedestal service.#2022-04-1101:21wilkerluciohello Joe, if you expose Pathom via some HTTP handler using the boundary interface at the border, that's all you need to connect to it via the HTTP connection feature on Pathom Viz#2022-04-1101:21wilkerluciothere is + button on top right of the app#2022-04-1101:21wilkerluciothe connector is more for development time, to connect to something in staging/prod you can use the HTTP path#2022-04-1303:20Joe R. SmithThanks Wilker-- should tracing and introspection work?#2022-04-1313:37wilkerlucioyes#2022-04-1414:20Joe R. SmithIt looks like my issue is my transit body interceptor isn't able to serialize resolvers correctly during index fetch. I'm seeing errors like this:
{:type java.lang.RuntimeException
   :message "java.lang.Exception: Not supported: class io.crescentinvest.api.pathom$user_count__56506"
   :at [com.cognitect.transit.impl.WriterFactory$1 write "WriterFactory.java" 65]}
do I need to register a transit write handler / use a special transit writer?
#2022-04-1414:32wilkerlucioyes, there is a transit namespace on pathom with read and write handlers for the custom map types#2022-04-1414:33wilkerlucioexample at: https://pathom3.wsscode.com/docs/tutorials/serverless-pathom-gcf#ring-handler-setup#2022-04-1414:34Joe R. Smiththanks-- figured out the write handlers, no more errors, but "refresh indexes" doesn't show any introspection results.#2022-04-1414:34Joe R. SmithI'll check out that tutorial, thanks#2022-04-1414:35Joe R. Smithahh, probably not encoding metadata šŸ™‚#2022-04-1121:32Ben GrabowHas anyone explored the idea of describing/validating the shape of inputs/outputs of resolvers using clojure spec/malli/etc? It looks like I could define my own plugin that applies to resolver calls (https://pathom3.wsscode.com/docs/plugins#pcrwrap-resolve), looks at the input/output keys, looks up specs in a registry, and validates the inputs/outputs for each resolver. Is there any prior art in this space?#2022-04-1121:35Ben GrabowIf I don't validate the input/output values at runtime, then I could publish whatever schema I want and my users would be left guessing what happens at runtime. Giving my users some peace of mind that the values do match the schema I publish would be a big win. The second level would be to advertise each key's schema in Pathom Viz.#2022-04-1121:41wilkerlucioyes, I played with that, I made a small library that I called eql-schemas, which basically does the validation if a DS based on the shape + specs: https://gist.github.com/wilkerlucio/71d2ac918e6dc116576b64b1f506c58d#2022-04-1121:43Ben GrabowThanks!#2022-04-1121:44Ben GrabowDo you have any thoughts on the best approach? I can think of two ways to guarantee coverage of the entire process: • Spec the query, and spec the output of each resolver • Spec the input of each resolver, and the output of each resolver The second way will involve a lot of duplicated work as data flows out of one resolver (validated) and flows into another resolver (validated again). It does not require parsing the EQL query though, and works with smart maps.#2022-04-1121:45Ben GrabowAnother question: Did you ever try creating a Pathom plug-in for this?#2022-04-1121:46Ben GrabowWhich extension point would you use?#2022-04-1121:46wilkerlucioI never done, I would avoid it in production because the overhead, but could be something useful to assist during development, making a plugin for it is quite easy, using wrap-resolve you can check both the input/output of resolvers and apply on it#2022-04-1121:46wilkerluciothis entry point: https://pathom3.wsscode.com/docs/plugins#pcrwrap-resolve#2022-04-1121:48wilkerlucionote this library I sent to you uses a different optional flag (the reason in my case is because my usage for it was outside pathom), you can either change the eql-schema code ot use the same optional attribute as Pathom, or you have to convert it on the plugin (to allow for optional things)#2022-04-1122:10wilkerlucio(or in short, replace every ::optional? there with ::pco/optional?)#2022-04-1210:19Bjƶrn Ebbinghaus@UANMXF34G I have a plugin for this. I add specs to mutations.
(def spec-plugin
  {::p/wrap-mutate
   (fn [mutate]
     (fn [env sym params]
       (if-let [spec (get-in env [::pc/indexes ::pc/index-mutations sym ::s/params])]
         (if (s/valid? spec params)
           (mutate env sym params)
           (do
             (log/debug (s/explain spec params))
             ;; TODO Errors are data too!
             (throw (ex-info "Failed validation!" (s/explain-data spec params)))))
         (mutate env sym params))))})
https://github.com/hhucn/decide3/blob/master/src/main/decide/server_components/pathom.clj#L49 And then:
(defmutation add-participant [env {user-id ::user/id, slug ::process/slug}]
  {::pc/params [::process/slug ::user/id]
   ::pc/output [::process/slug]
   ::s/params (s/keys :req [::process/slug ::user/id])})
It is relatively old and more of an experiment that stuck. You can probably improve it.
#2022-04-1213:56Ben GrabowHere's what I came up with:
(defn key-errors
  [schema-registry [k v]]
  (let [schema (get schema-registry k)]
    (if schema
      (and
        (not (mc/validate schema v))
        (me/humanize (mc/explain schema v)))
      (timbre/warnf "Missing schema for key %s" k))))

(defn collect-key-errors
  [schema-registry m]
  (->> (for [[k v] m
             :let [errors (key-errors schema-registry [k v])]]
         (when errors [k errors]))
       (remove nil?)
       (into {})
       not-empty))

(def validate-resolver-inputs
  {::p.plugin/id ::validate-resolver-inputs
   ::pcr/wrap-resolve
   (fn wrap-validate-resolver-inputs
     [inner-fn]
     (fn wrapped-validate-resolver-inputs [env input]
       (let [registry (:metosin.malli/registry env)]
         (if-let [key-errors (collect-key-errors registry input)]
           (throw (ex-info
                    (format "Error validating resolver input: %s" {:input-keys (keys input)
                                                                   :key-errors key-errors})
                    {:input-keys (keys input)
                     :key-errors key-errors}))
           (let [output (inner-fn env input)]
             (if-let [key-errors (collect-key-errors registry output)]
               (throw (ex-info
                        (format "Error validating resolver output: %s" {:output-keys (keys output)
                                                                        :key-errors  key-errors})
                        {:output-keys (keys output)
                         :key-errors  key-errors}))
               output))))))})

(def pathom-index
  (-> {:metosin.malli/registry {:some.ns/attr [:map [:foo :string][:bar :nat-int]]}}
    (p.plugin/register [pathom-util/validate-resolver-inputs])
    (pci/register resolvers)))
My assumption is the schema registry will be keyed on pathom attribute keys. The defresolver form doesn't need anything extra in it for this to work, but this design also assumes that all resolvers will want their inputs and outputs validated. I like the idea of attaching metadata to the resolver/mutation directly to say which parts should be validated. That would be useful for more performance-sensitive usages, or when some very specific attrs are causing problems or have mission-critical shape.
#2022-04-1213:57Ben Grabow> ;; TODO Errors are data too! I love this. I'm going to start using this comment in my code too. šŸ˜†#2022-04-1214:50wilkerlucionice šŸ™‚ just one thing, ::p.plugin/id is expected to be a symbol, I should add some validation to check it, but if you run with guardrails on it may complain (because the spec of plugin id is a symbol)#2022-04-1220:20Ben GrabowInteresting! I noticed it was a symbol in the docs but I'm curious why symbols are a better fit here than keywords.#2022-04-1303:09Brett RowberryI'm just getting started learning about Pathom after listening to a few podcasts with Wilker as a guest published in 2020. How do people feel today about using Pathom 3 for new development?#2022-04-1304:57jmayaalvWe have been using pathom3 in production already for about 6 months and feel really happy about it. We didn't have any major problems. Performance is top notch, no major bugs or breaking changes. Only problem is that now clients are getting used to quick delivery times šŸ˜…#2022-04-1304:58nivekuilthe biggest problem with pathom is that once you've tried it, it is just painful to program without it#2022-04-1315:11Brett RowberryThanks, that's good enough for me!#2022-04-1319:08wilkerlucioglad to hear you got to those, there is a new podcast that was released last week as well, in case you want get more of those šŸ™‚ https://cognitect.com/cognicast/168#2022-04-1315:46sheluchinIs there a way to express that resolver x is the same thing as resolver y but with added params? Something to avoid reproducing the logic but still provide a new name. e.g. sub-rand is the same thing as sub {:n (rand)}.
::pco/input [`(:sub {:n ~(rand)})]
gets pretty close but that's compiled in and so the input isn't dynamic.
#2022-04-1316:00markaddlemanDo optional inputs get you want you want? https://pathom3.wsscode.com/docs/resolvers/#optional-inputs#2022-04-1316:02markaddlemanIf I understand your use case correct, you would have a single single resolver that optionally takes {:sub [:n]}#2022-04-1316:02sheluchinI don't think so. I still want to have a separate name for the output attributes.#2022-04-1316:02markaddlemanAh, I didn't catch that#2022-04-1316:07sheluchinIt's obviously a simplified example, but the long and short of it is that I want one resolver to contain a bunch of logic and some other resolvers to make use of that resolver by just parameterizing the logic within it. Like now->time-ago would be the flexible one, and then there'd be a now->years-ago and now->months-ago which would use the first one but fill in the :time-unit param and adds their own n.#2022-04-1316:13markaddlemanIn that case, I think your best bet is to factor out the common code to a regular clojure fn and call that fn from the different results.#2022-04-1316:14markaddlemanYou may be able to take advantage of ::pco/op-name from the defresolver macro#2022-04-1316:14sheluchinI suspect you're right, but given that there are alias resolvers and other helpers that seem to get pretty close, I thought there might be a more Pathomic way.#2022-04-1316:16markaddlemanI've often wished for more defresolver attributes to perform more logic but I end up realizing it's just better to drop to regular clojure fn.#2022-04-1316:16markaddlemanOne thing that does come in handy: resolvers implement IFn so you can use them like regular fns#2022-04-1316:17sheluchinHmm, that might help here.#2022-04-1316:30sheluchin@U2845S9KL good idea. Thanks for the tip:
(comment
 (pco/defresolver sub
   [env _]
   {::pco/output [:sub]}
   {:sub (- 100 (or (:n (pco/params env)) 0))})

 (pco/defresolver sub-rand
   [_]
   {::pco/output [:sub-rand]}
   {:sub-rand (:sub (sub {::pcp/node {::pcp/params {:n (rand)}}} :_))})

 (p.eql/process (pci/register [sub sub-rand])
                [:sub-rand]))
#2022-04-1319:06wilkerluciothis is an interesting thing to think about, how to transfer params over. one simple way is to use inputs instead of params, since they already flow. this makes me think that if you alias an attribute, the aliased version wont get any of the params used from the source attribute#2022-04-1319:07wilkerlucio(also, quite confusing to read this thread when you both have the same profile pic, rsrs)#2022-04-1319:52sheluchin@U066U8JQJ do you consider invoking resolvers directly like that as an antipattern? The docs mention direct invocation is mostly used for testing.#2022-04-1319:54wilkerlucionot really, but you lose the auto-caching mechanism when you do it (also plugin invokation and etc, but in this case that may be desired)#2022-04-1319:54wilkerluciousing a resolver in this way is pretty much as same as calling a regular function (which wont leverage any of the pathom running stuff)#2022-04-1319:55wilkerluciothe situation makes me think that a proper way to transfer params could be nice#2022-04-1320:08sheluchinIt sounds like param routing is a recurring question in this stage of Pathom's development. I recall something similar coming up recently... Maybe there's a PR or something I read. Thanks for the feedback.#2022-04-1320:32wilkerlucioI dont remember the other occasion, can you please refer to me if you find it?#2022-04-1320:36sheluchin@U066U8JQJ maybe this is what I'm thinking about: https://clojurians.slack.com/archives/C87NB2CFN/p1645491879887139?thread_ts=1645478052.269719&amp;cid=C87NB2CFN Is it the same idea about forwarding params?#2022-04-1320:37wilkerluciono, that's a different thing, this other one is related to transfering sub-queries to dynamic resolvers#2022-04-1320:43sheluchinHmm, but you do specifically mention in that post that in addition forwarding sub-query parts, thinking about forwarding params could be another side of it. I'm probably missing some nuance in the language and/or implementation. No worries. Thanks again @U066U8JQJ.#2022-04-1320:45wilkerlucioto help clarify, the params I mention is for making possible to transfer the sub-query, because its not possible for Pathom to figure it out where it should append the sub-query on the target part, so the user must mark in the query where the sub-query should be injected, thats where the param comes in, in this case its a flag to help pathom (more like we use params to mark parts of the query as optional)#2022-04-1320:51sheluchinOh, ok. So more like a request configuration flag than the params we pass to resolvers to parameterize their internal behaviour. I guess it's like a meta param :)#2022-04-1320:52wilkerluciowe can say so šŸ™‚#2022-04-1319:08wilkerlucioglad to hear you got to those, there is a new podcast that was released last week as well, in case you want get more of those šŸ™‚ https://cognitect.com/cognicast/168#2022-04-1320:17Benjamin(cross-posted from #fulcro, as it has to do with pathom) Is there a proper way to get the pathom env so I can run server-initiated mutations or queries? Digging through fulcro RAD's form code I found this:
(def save-form
     {:com.wsscode.pathom.connect/mutate (fn [env params] (save-form* env params))
      :com.wsscode.pathom.connect/sym    `save-form
      :com.wsscode.pathom.connect/params #{::id ::master-pk ::delta}})
Which seems to translate somehow into a callable function that calls the passed mutate fn with the env, but I haven't yet figured out where or how this happens. Edit: Solved! Turns out you can create your own env with whatever you need. Although I am still a little curious about how save-form works. :)
#2022-04-1322:35tony.kayIn Pathom 2 resolvers and mutations are just maps. That is the definition of the mutation, which is then added to the pathom processor.#2022-04-1322:36tony.kaySee https://book.fulcrologic.com/RAD.html#_pathom_parser#2022-04-1322:37tony.kayform/resolvers is added in the call to new-parser (which includes the save mutation)#2022-04-1322:38tony.kayThat generic save-form function then looks in env for database adapters, and passes along the diff for actual saving…so for example the datomic/pathom-plugin and form/pathom-plugin each add elements that are needed by that function to do the actual work.#2022-04-1323:33BenjaminAh, that makes sense. Thank you!#2022-04-1411:40aratareHi @wilkerlucio, I'm having some params-related problem and I'm not sure what is causing it. I have a resolver that is expecting some params sent up by fulcro, and when I checked (-> env :ast :params) it is nil despite the param was sent up normally. I'll put the code snippets in this thread for illustration.#2022-04-1411:41aratareParser:
(p/parser {::p/env     {::p/reader                 [p/map-reader pc/reader2 pc/ident-reader pc/index-reader]
                          ::p/process-error          process-error
                          ::pc/mutation-join-globals [:tempids]}
             ::p/mutate  pc/mutate
             ::p/plugins [(pc/connect-plugin {::pc/register resolvers})
                          p/error-handler-plugin
                          p/trace-plugin
                          ;; FIXME: Seems to be fired twice
                          (p/post-process-parser-plugin (fn [output]
                                                          (let [output (-> output p/elide-not-found p/raise-errors)]
                                                            (if-let [errors (::p/errors output)]
                                                              errors
                                                              output))))]})
#2022-04-1411:42aratareResolver:
(defresolver bookmarks-resolver
  [{{user-id :user/id} :request :as env} {:tab/keys [id] :as input}]
  {::pc/input  #{:tab/id}
   ::pc/output [{:tab/bookmarks bookmark-output}]}
  ...)
#2022-04-1411:42aratareRequest (as shown in fulcro inspect):#2022-04-1411:42aratare
[({[:tab/id #uuid "bc1599de-e58a-4d3b-b61b-b0e50b6a2749"]
   [{:tab/bookmarks
     [:bookmark/id
      :bookmark/title
      :bookmark/url
      :bookmark/favourite
      :bookmark/image
      :bookmark/tab-id
      {:bookmark/tags [:tag/id :tag/name :tag/colour]}]}]}
  {:tab/password "v"})]
#2022-04-1411:43aratareAs you can see I'm sending up :tab/password but it doesn't exist in (-> env :ast :params)#2022-04-1411:46aratareIs there any special setup I'd need to do to be able to retrieve params?#2022-04-1417:44wilkerluciothe problem here is the param location, because Fulcro injects the params to the ident, not to the attributes, so Pathom has it in a different location than you are expecting. there two ways around it, the most common seems to be to have a plugin to propagate params down, or you have to set the params in the query itself (instead of using the`:params` option from Fulcro)#2022-04-1417:44wilkerlucioI dont have it now, but if you ask on #fulcro somebody probably has that code already to give to you#2022-04-1421:28nivekuil
{::p.eql/wrap-process-ast    (fn [process]      (fn [env ast]        (-> (if (:query-params env)              env              (assoc env :query-params                     (reduce                      (fn [qps {:keys [type params] :as x}]                        (cond-> qps                          (and (not= :call type) (seq params)) (merge params)))                      {}                      (:children ast))))            (process ast))))}
#2022-04-1421:29nivekuiloh, mine is the RAD one ported to pathom3#2022-04-1500:42aratare@wilkerlucio Yep indeed a plugin is needed for this. Thank you and @U797MAJ8M for the help. It's working wonderfully now šŸ™‚#2022-04-1900:04aratareHi @wilkerlucio, if I have a query like this [{:user/tabs [{:tab/bookmarks [:bookmark/id]}]}] and if the resolver that handles :bookmark/id, or any resolver at any level in the query, throws an error, currently Pathom will handle this by utilising ::p/errors for the failed attribute/field, and everything else will proceed as normal. In this case I don't want that, instead I want the entire query to fail and return some custom error like {:user/tabs {:error true :error-message "failed"}}. There's a :fail-fast? parser option but I can't seem to catch and process it. Previously I can achieve this behaviour by using mutations + ::p/process-error, but since semantically it's wrong to use mutations to fetch data, I'm migrating everything to resolvers but currently stuck. What is the best way to achieve this? Thank you in advance.#2022-04-1901:04wilkerlucioI guess you are using lenient mode right? because on strict mode, any fail will fail everything#2022-04-1901:05aratareAre you referring to Pathom3?#2022-04-1901:05wilkerlucioah, yes#2022-04-1901:05aratareIn which case I’m still on P2#2022-04-1901:05aratareIs there a way to do this on P2?#2022-04-1901:05wilkerluciocan you make a working example so we can talk over? with a full example, what happens, and what you would like to happen#2022-04-1901:08aratareI do have a working example, which is my current project but it’s bit bulky. Let me put up a small example with some resolvers.#2022-04-1901:12aratareOk I’m not on my home PC atm so I can’t create a project fast enough. I do have some code to show you what I have.#2022-04-1901:12aratareHere is the first resolver of interest: https://github.com/aratare-jp/shinsetsu/blob/main/src/clj/shinsetsu/resolvers/tab.clj#L31#2022-04-1901:13aratarepretty simple, it gets the user id then retrieves all the tabs from such user#2022-04-1901:14aratarelater on in my app flow, I need to fetch all the bookmarks within some tab, so I have another resolver for this: https://github.com/aratare-jp/shinsetsu/blob/main/src/clj/shinsetsu/resolvers/bookmark.clj#L29#2022-04-1901:14aratarethere are 2 types of tabs, one protected and one public (unprotected)#2022-04-1901:15aratareimma ignore the public#2022-04-1901:15aratarefor protected tab, I need to submit a password to fetch all the bookmarks within such tab#2022-04-1901:15aratareso if I submit the wrong password, you’d expect an error saying it’s the wrong password#2022-04-1901:17aratarebut currently if I fetch a tab on some of its fields, and join with the bookmarks, when the password is incorrect only :tab/bookmarks will have a reader-error#2022-04-1901:18aratareall the other tab fields will resolve normally#2022-04-1901:19aratareI want to have the all-or-nothing behaviour of P3 where if I can’t fetch the bookmarks due to wrong password, an error is returned instead#2022-04-1901:19aratarepreviously I could achieve this by using mutations, i.e. fetch-tabs and fetch-bookmarks.#2022-04-1901:20wilkerlucioif you have multiple tabs, you want to fail all the tabs, or just the tabs which the user is not allowed to access?#2022-04-1901:21aratareall of them#2022-04-1901:21wilkerlucioa way to solve this via query design is to change your query to allow for a layer controlling the access, like: [{:user/tabs [{:tab/allowed-tab [{:tab/bookmarks [:bookmark/id]}]}]}]#2022-04-1901:22wilkerluciothis way, you can fail at :tab/allowed-tab, and them both bookmarks and all the siblings will not attempt to run#2022-04-1901:22wilkerluciothis is how I would do it in this case#2022-04-1901:23aratareinteresting. I haven’t thought of that.#2022-04-1901:24wilkerlucioyou can make what you want by having a plugin wrapping around the entity process, but feels flacky, would require some very specific code (like checking for a presence of bookmarks on the result) and add unescessary overhead (since it would have to check every entity)#2022-04-1901:24wilkerlucioso I would go with :tab/allowed-tab šŸ™‚#2022-04-1901:25aratare> you can make what you want Are you referring to the custom error?#2022-04-1901:26wilkerlucioyeah, making everything fail given a specific circumstance in the result#2022-04-1901:27wilkerluciobut would be ugly, hehe#2022-04-1901:29aratareI’ll give allowed-tab a go. Also I tried :fail-fast? and it did give me what I wanted, i.e. error thrown -> back track all the way to the top. But where can I catch the exception thrown?#2022-04-1901:29wilkerlucioI'm not sure if fail-fast will work as expected in this case#2022-04-1901:29wilkerlucioits intended to be used at the root, so errors are thrown immediatly (mostly useful for development environments, to see error earlier)#2022-04-1901:31aratareyep which fits perfectly into my usecase interestingly#2022-04-1901:31aratarefrom the snippet I found on Pathom docs
(parser {::p/fail-fast? true}
        [{:go [:key {:nest [:trigger-error :other]}
               :trigger-error]}])
; => CompilerException clojure.lang.ExceptionInfo: Error triggered {:foo "bar"}, ...
#2022-04-1901:32aratareI want that exception but even with a try-catch around the parser it just does the dont-mind-me-im-just-passing-by move on me didn’t get caught for some reason šŸ˜…#2022-04-1901:34wilkerlucionot sure if I get what you saying now, if you wrap on a try/catch you can't get this exception?#2022-04-1901:35wilkerlucioI think I replicated the issue, see if looks like what you see:#2022-04-1901:35wilkerlucio
(ns test.demo-fail-sublings
  (:require [com.wsscode.pathom.core :as p]
            [com.wsscode.pathom.connect :as pc]))

(pc/defresolver user-tabs []
  {:user/tabs
   [{:tab/id          1
     :tab/accessible? true}
    {:tab/id          2
     :tab/accessible? false}]})

(pc/defresolver bookmarks [{:tab/keys [id accessible?]}]
  {:tab/bookmarks
   (if accessible?
     [{:bookmark/id id}]
     (throw (ex-info "Fail here" {})))})

(pc/defresolver tab-stuff [{:tab/keys [id]}]
  {:tab/name (str "Tab " id)})

(def registry
  [user-tabs
   bookmarks
   tab-stuff])

(def parser
  (p/parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/reader2
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate
     ::p/plugins [(pc/connect-plugin {::pc/register registry})
                  p/error-handler-plugin
                  p/trace-plugin]}))

(comment
  (parser {}
    [{:user/tabs
      [{:tab/bookmarks
        [:bookmark/id]}
       :tab/name]}])
  ; =>
  ; {:user/tabs [{:tab/bookmarks [{:bookmark/id 1}], :tab/name "Tab 1"}
  ;             {:tab/bookmarks :com.wsscode.pathom.core/reader-error, :tab/name "Tab 2"}],
  ; :com.wsscode.pathom.core/errors {[:user/tabs 1 :tab/bookmarks] "class clojure.lang.ExceptionInfo: Fail here - {}"}}
  )
#2022-04-1901:36aratareYep it’s perfect.#2022-04-1901:39wilkerluciosolution using :tab/accessible:
(ns test.demo-fail-sublings
  (:require [com.wsscode.pathom.core :as p]
            [com.wsscode.pathom.connect :as pc]))

(pc/defresolver user-tabs []
  {:user/tabs
   [{:tab/id          1
     :tab/accessible? true}
    {:tab/id          2
     :tab/accessible? false}]})

(pc/defresolver bookmarks [{:tab/keys [id accessible?]}]
  {:tab/bookmarks
   (if accessible?
     [{:bookmark/id id}]
     (throw (ex-info "Fail here" {})))})

(pc/defresolver tab-accessible [env {:tab/keys [accessible?]}]
  {::pc/input #{:tab/id :tab/accessible?}}
  {:tab/accessible
   (if accessible?
     (p/entity env)
     (throw (ex-info "Fail here" {})))})

(pc/defresolver tab-stuff [{:tab/keys [id]}]
  {:tab/name (str "Tab " id)})

(def registry
  [user-tabs
   bookmarks
   tab-stuff
   tab-accessible])

(def parser
  (p/parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/reader2
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate
     ::p/plugins [(pc/connect-plugin {::pc/register registry})
                  p/error-handler-plugin
                  p/trace-plugin]}))

(comment
  (parser {}
    [{:user/tabs
      [{:tab/accessible
        [{:tab/bookmarks
          [:bookmark/id]}
         :tab/name]}]}])
  ; =>
  ; {:user/tabs [{:tab/accessible {:tab/bookmarks [{:bookmark/id 1}], :tab/name "Tab 1"}}
  ;              {:tab/accessible :com.wsscode.pathom.core/reader-error}],
  ; :com.wsscode.pathom.core/errors {[:user/tabs 1 :tab/accessible] "class clojure.lang.ExceptionInfo: Fail here - {}"}}
  )
#2022-04-1901:42aratareOK will give it a go. May be i’ve been approaching this problem from a completely wrong angle all along.#2022-04-1901:42aratarešŸ˜…#2022-04-1901:42wilkerluciothis is all very novel stuff, we are all still figuring it out šŸ˜…#2022-04-1901:43wilkerluciono books on information modelling though attributes out there, yet šŸ˜›#2022-04-1901:45aratarešŸ‘:skin-tone-3:#2022-04-1901:45aratareThis problem has made me want to migrate to P3 all the more šŸ˜…#2022-04-1901:46wilkerlucioI don't think it would solve this one for you, because strict mode will break the whole query all the time, like fail-fast in this sense#2022-04-1901:46wilkerluciono way to apply this strictness to a sub-part#2022-04-1901:46aratareSo for this project I’m looking for absolute all-or-nothing#2022-04-1901:47aratareIf one part fails, nothing gets returned except an error#2022-04-1901:47wilkerluciothen what about:
(parser {::p/fail-fast? true}
    [{:user/tabs
      [{:tab/bookmarks
        [:bookmark/id]}
       :tab/name]}])
Execution error (ExceptionInfo) at test.demo-fail-sublings/bookmarks (demo_fail_sublings.clj:16).
Fail here
#2022-04-1901:48wilkerluciowhat reader are you using?#2022-04-1901:48arataregive me a sec#2022-04-1901:49aratare[pc/open-ident-reader p/map-reader pc/reader2 pc/index-reader]#2022-04-1901:49wilkerlucioso the fail-fast should work#2022-04-1901:50wilkerluciowhat dont-mind-me-im-just-passing-by move on me means?#2022-04-1901:50wilkerlucio
(try
   (parser {::p/fail-fast? true}
     [{:user/tabs
       [{:tab/bookmarks
         [:bookmark/id]}
        :tab/name]}])
   (catch Throwable ex
     (println "Gotcha" (ex-message ex))))
Gotcha Fail here
=> nil
#2022-04-1901:50aratareYep it does work as expected, but when I put the try-catch around it, the catch part didn’t get fired somehow. That’s what I meant.#2022-04-1901:51aratarePerhaps I had some wanky code in there at the time#2022-04-1901:51aratareImma try it again.#2022-04-1901:51wilkerluciofull demo with catch:#2022-04-1901:51wilkerlucio
(ns test.demo-fail-sublings
  (:require [com.wsscode.pathom.core :as p]
            [com.wsscode.pathom.connect :as pc]))

(pc/defresolver user-tabs []
  {:user/tabs
   [{:tab/id          1
     :tab/accessible? true}
    {:tab/id          2
     :tab/accessible? false}]})

(pc/defresolver bookmarks [{:tab/keys [id accessible?]}]
  {:tab/bookmarks
   (if accessible?
     [{:bookmark/id id}]
     (throw (ex-info "Fail here" {})))})

(pc/defresolver tab-stuff [{:tab/keys [id]}]
  {:tab/name (str "Tab " id)})

(def registry
  [user-tabs
   bookmarks
   tab-stuff])

(def parser
  (p/parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/reader2
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate
     ::p/plugins [(pc/connect-plugin {::pc/register registry})
                  p/error-handler-plugin
                  p/trace-plugin
                  {::p/wrap-parser
                   (fn [parser]
                     (fn [env tx]
                       (tap> tx)
                       (parser env tx)))}]}))

(comment
  (try
   (parser {::p/fail-fast? true}
     [{:user/tabs
       [{:tab/bookmarks
         [:bookmark/id]}
        :tab/name]}])
   (catch Throwable ex
     (println "Gotcha" (ex-message ex))))
  ; Gotcha Fail here
  ; => nil
  )
#2022-04-1901:52wilkerluciocause not being able to catch sounds strange#2022-04-1901:52aratareThen I definitely had some wanky code#2022-04-1901:53aratareI’m glad this is just me messing up all along šŸ˜…#2022-04-1901:54aratareThanks for the extensive help @wilkerlucio Will definitely try both of them šŸ™‚#2022-04-1901:54wilkerluciohave fun šŸ™‚#2022-04-1901:55aratarefun is mandatory šŸ˜›#2022-04-2014:41Joe R. SmithGood morning Pathomites: I'm getting: java.lang.RuntimeException: No such var: pcp/compute-dynamic-resolver-nested-requirements with pathom3-datomic rev fec7c00384ce6b472c196e51661e0e3a6048d314 originating here: https://github.com/wilkerlucio/pathom3-datomic/blob/bad83ee5a5973a259b2afad766fc043ab91e6b99/src/main/com/wsscode/pathom3/connect/datomic.clj#L158 That function doesn't appear to exist in Pathom3.#2022-04-2014:42wilkerluciohello, what Pathom 3 version are you using?#2022-04-2014:47Joe R. SmithI'm using the transitive dependency from that version of pathom-datomic,#2022-04-2014:49Joe R. Smith(looking it up now)#2022-04-2014:49wilkerluciotry using all the latest#2022-04-2014:49wilkerlucioPathom just released, and bad83ee5a5973a259b2afad766fc043ab91e6b99 for pathom3-datomic#2022-04-2014:51Joe R. Smithsame error with:
com.wsscode/pathom3        {:git/url ""
                                         :sha "d52e431223c3dde193ab07a7bd4dda6c2a24edf7"}

             com.wsscode/pathom3-datomic {:git/url ""
                                          :sha     "bad83ee5a5973a259b2afad766fc043ab91e6b99"}
#2022-04-2014:52wilkerlucioPathom 3 has a maven version, you can use 2022.04.20-alpha#2022-04-2014:52wilkerluciobut still, maybe its a legit bug, let me look closer#2022-04-2014:53Joe R. SmithSame issue.#2022-04-2015:03wilkerlucio@U087E7EJW can you try pathom3-datomic 9dfa5838b8ae9b5a7caa5d3a4d2eae5bdadc6373 please?#2022-04-2015:07wilkerlucio(just pushed a fix on it)#2022-04-2015:10Joe R. Smiththat fixed it!#2022-04-2015:10Joe R. Smiththank you#2022-04-2015:18wilkerluciothanks for reporting šŸ™#2022-04-2014:44wilkerlucio#2022-04-2115:31Bjƶrn EbbinghausHave you considered dropping the -alpha suffix? Every time someone asks if it is safe to migrate to v3 the answers are: "Everything from v2 is working in v3" "We use v3 in production for the last x month, without issue"#2022-04-2115:53wilkerlucioyeah man, from time to time I think about it, my lack of confidence at this stage is more around dynamic resolvers, there still some rough edges there, but maybe that can be addressed with more specific indications#2022-04-2118:05nivekuilwhat are people testing dynamic resolvers for right now? I'd like to help but can't think of a use case and I think I use pathom pretty extensively#2022-04-2118:06nivekuilbut you could always mark it as stable with some features in alpha#2022-04-2118:06wilkerluciodynamic resolvers are now used for GraphQL integration, but most important for me is to integrate Pathom with other Pathom instances, enabling a distributed processing architecture#2022-04-2118:07nivekuilyeah, it's a really cool idea but what problem do you see it solving today#2022-04-2118:07nivekuiltypically those would be exposed as graphql APIs anyway right#2022-04-2118:08wilkerlucioone simple and interesting way I think to try it is to design the system to have 2 Pathom, one in the server, one in the client, this allows you to delegate some processing parts to the client#2022-04-2118:09nivekuilah right, forgot I was meaning to use that to build a caching layer for fulcro#2022-04-2118:09wilkerluciopushing further, a 3 pathom instances could go: Web <-> Web Worker <-> Server, this allows for choosing that specific portions should be computed at the web worker to dont freese the UI#2022-04-2118:09nivekuila little worried about bundle size/initial js load time with that#2022-04-2118:10wilkerlucioand the most interesting for me is when dealing with micro-services, making each service provide its own graph, so you could plug in multiple graphs at once, while allow teams to develop them independently#2022-04-2118:12wilkerluciowhy the worry on size / initial load? the webworker is an extra load that can go in parallel with the main page#2022-04-2118:14nivekuilI haven't thought about web workers actually, it is interesting. currently I am not using pathom in the client at all and I know pathom has a bit of cold overhead#2022-04-2121:23Ben GrabowIs it possible to shim the experimental features out of the main repo in a way that can be easily reintegrated once those experimental features are more stable? The core shouldn't have to appear unstable just because of some optional features still under observation. Reitit is a good example of a lib that does this subdivision thing well: https://github.com/metosin/reitit/tree/master/modules#2022-04-2513:30wilkerlucionot in this case @UANMXF34G, because the dynamic resolvers are a core part of the processing and have tight integration with the rest (I can't split the dynamic parts, its a facet of the process)#2022-04-3017:39normanI would feel much more comfortable in upgrading my pathom2 app to pathom3 if it weren't alpha.#2022-04-2022:03Joe R. SmithUsing pathom3-datomic– I've got it all wired up and indexes are populated in pathom-viz, but getting errors in join queries like this: :user/date-registered is, indeed, an attribute, and it shows up in the indexes.#2022-04-2105:15wilkerluciothis looks like a missing setup of ident attributes, can you give an example of what this query would look like in GraphQL? this way we can figure how to make the ident attributes settings#2022-04-2513:27wilkerlucioah, sorry, I confused graphql with datomic, there are not ident attributes on datomic, can you make a repro caso of what you are experiencing?#2022-05-0301:46Matt SecoskeHi @wilkerlucio! I’ve reproduced the issue, see here: https://github.com/secos/pathom3-datomic/commit/0c4dafb5e47046b9450075b1e73357ced4cecbec#2022-05-0323:10Joe R. Smith^ @wilkerlucio this is our repro using Datomic dev-local (client-api)#2022-05-0323:11wilkerluciothanks for the repro, hope to look at it soon, just been a busy week :)#2022-05-0323:11Joe R. Smiththank you!#2022-05-0501:50wilkerluciohello, I just had a look in the repro, but I notice the changed example is starting from an empty database, the original tests connect to the mbrainz database (per this line: https://github.com/secos/pathom3-datomic/commit/0c4dafb5e47046b9450075b1e73357ced4cecbec#diff-682efd186f370fa01384ae9ce43d3091d9cf2c63d71d9e2ddb25b020c7155e78L15), so without connecting to the mbrainz its expected that all the tests fail, because they assume we will have the mbrainz schema/data there for testing#2022-05-0513:06Matt Secoskefacepalm#2022-05-0513:06Matt SecoskeMy apologies for missing that! I’ll get it sorted.#2022-05-0922:03Matt SecoskeHi @wilkerlucio, finally got back into this. I’ve ACTUALLY reproduced it now, and (somewhat) simplified the test case… one does need to set up dev-local and the datomic-samples data before the test will run. https://github.com/secos/pathom3-datomic/commit/534d9ef29c96e8deb917a6adb369fa747780ba0c#2022-05-1011:50wilkerlucioawesome, thanks, gonna check it out#2022-05-2702:27wilkerlucio@U03B7KALC3Y sorry the long time on this one, I merged this commit to main and it should fix the ident lookup: https://github.com/wilkerlucio/pathom3-datomic/pull/3/commits/8104c727f79f1ad52d2a9b8aaeac350f5e7c8bd0#2022-05-2702:27wilkerluciothis made your test case pass, please let me know if you still have issues#2022-05-2713:49Matt SecoskeThank you @wilkerlucio. Will take a look!#2022-05-3116:27Matt SecoskeThat worked @wilkerlucio - thank you so much!#2022-05-0301:46Matt SecoskeHi @wilkerlucio! I’ve reproduced the issue, see here: https://github.com/secos/pathom3-datomic/commit/0c4dafb5e47046b9450075b1e73357ced4cecbec#2022-05-0922:03Matt SecoskeHi @wilkerlucio, finally got back into this. I’ve ACTUALLY reproduced it now, and (somewhat) simplified the test case… one does need to set up dev-local and the datomic-samples data before the test will run. https://github.com/secos/pathom3-datomic/commit/534d9ef29c96e8deb917a6adb369fa747780ba0c#2022-04-2513:26wilkerlucio#2022-04-2613:14sheluchinI'm looking at ways to manage my ETL pipeline. A common solution from libraries like https://domino-clj.github.io/ and https://github.com/commsor/titanoboa is to define the parts of the pipeline in a dependency graph and follow that graph to perform the steps of the pipeline. I think there is some potential to use Pathom in this way. We already define a data dependency graph when using it. On the other hand, I know Pathom adds some overhead and this might make it difficult for a very large number of records, as ETL often includes, but there is that ::pco/final that maybe be of some help. Does anyone know if using Pathom for ETL like this has been attempted somewhere?#2022-04-2614:31wilkerlucioI've considered that, I believe its possible to leverage the planner in Pathom to have the schematics on what to run, but for the runner it would do something very different, maybe generating spark statements or something#2022-04-2620:15sheluchinI'm not familiar with spark statements but I'll read up about it. Indeed, maybe just using the planner could be a good step in the right direction.#2022-04-2616:05Ī»ustin f(n)I am updating from pathom3 version 2022.02.01-1-alpha to 2022.03.17-alpha, 2022.04.20-alpha , but it is causing some existing mutation unit tests to fail with the message
ERROR in (create-comment-test) (planner.cljc:474)
Uncaught exception, not in assertion.
expected: nil
  actual: java.lang.AssertionError: Assert failed: Tried to remove node 24 that still contains references pointing to it. Move
      the run-next references from the pointer nodes before removing it. Also check if
      parent is branch and trying to merge.
(if node-parents (every? (fn* [p1__44092#] (not= node-id (get-node graph p1__44092# :com.wsscode.pathom3.connect.planner/run-next))) node-parents) true)
Is this a bug, or is it simply the new planner catching something odd we were doing from before that I could fix? How could I start digging in deeper to debug this?
#2022-04-2616:10Ī»ustin f(n)Ah. I should just use a more updated version, preferrably non-alpha šŸ˜…#2022-04-2616:19Ī»ustin f(n)Ah. Everything is still alpha. This same error happens for me on version 2022.04.20-alpha#2022-04-2619:48wilkerluciohello @U7Y7601B2, yep, all alpha still šŸ˜… can you give a me a repro? its possibly a regression, but need an example to check#2022-04-2621:13wilkerlucio(anyway, this error should never happen, its presence means there is something wrong in the planner algorithm)#2022-04-2704:44Ī»ustin f(n)Ah. Sounds worth extracting out a simple repro example from our system then.#2022-04-2718:59Ī»ustin f(n)Ok, I narrowed it down as much as I could.#2022-04-2718:59Ī»ustin f(n)
(ns repro
  (:require [clojure.test :refer :all]
            [com.wsscode.pathom3.connect.indexes :as pci]
            [com.wsscode.pathom3.interface.eql :as p.eql]
            [com.wsscode.pathom3.connect.operation :as pco]
            [com.wsscode.pathom3.connect.built-in.resolvers :as pbir]))

(pco/defresolver get-comment []
  {:comment/author {:user/id "user-id"}})

(def aliases (pbir/equivalence-resolver :comment/author :user))

(pco/defresolver user-resolver
  [{id :user/id}]
  {:user/avatar-filename "avatar-filename"})

(pco/defresolver avatar
  [{user             :user}]
  {::pco/input [{:user [:user/avatar-filename]}]}
  {:user/avatar user})

(pco/defresolver user-object-resolver
  []
  {::pco/output [{:user [:user/id]}]}
  {:user {:user/id "user-id"}})

(deftest repro-test
  (is
    (thrown?
      AssertionError
      (p.eql/process
        (pci/register [get-comment
                       aliases
                       user-resolver
                       user-object-resolver
                       avatar])
        {}
        [{:user
          [:user/avatar]}]))))
#2022-04-2719:00wilkerluciothanks, can you please open an issue in Pathom 3 repo (https://github.com/wilkerlucio/pathom3/issues)?#2022-04-2719:25wilkerlucio@U7Y7601B2 I think I understand already the bug, its a situation where a node must be removed, but the algorithm wasn't expecting the node to have parents in this case, and your repro demonstrate a case where it does happen#2022-04-2719:26wilkerlucioPathom is trying to remove the node author->user-alias because it notices that this path can't fulfill the nested requirements#2022-04-2719:27wilkerluciobut a node can't have parents when its removed, which is correct, so I think the way to go here is to remove the whole ancestor chain as well, this would translate in the user-object-resolver being the only valid path in this scenario#2022-04-2719:32Ī»ustin f(n)Ah. Well I finished submitting the case for you šŸ˜… https://github.com/wilkerlucio/pathom3/issues/136#2022-04-2719:57wilkerlucio@U7Y7601B2 pushed a fix to main, can you try so I can confirm the fix?#2022-04-2720:21Ī»ustin f(n)I am unfamiliar with how to bring in dependancies from an active branch rather than released :mvn/version could you point me to some docs or show me what a deps.edn import would look like?#2022-04-2720:21wilkerluciosure, one sec#2022-04-2720:22wilkerlucioyou can use this to import Pathom 3:
com.wsscode/pathom3        {:git/url ""
                              :sha     "28956c7f5d6dd259effc09567829c096932714a7"}
#2022-04-2720:24Ī»ustin f(n)Oof, I was so close to doing it right. Thanks!#2022-04-2720:35Ī»ustin f(n)@U066U8JQJ That fixed it. Unfortunately for me, there are 2 other unit tests of ours that fail on updating pathom3 that are apparently unrelated. Ones with less obvious errors thrown in my face. Looks like I need to make more repros#2022-04-2720:35wilkerluciothanks for bringing those, happy to keep debugging and tacking those with you#2022-04-2620:46nharschHi. I’m just getting started with pathom and have a pretty basic question. I have the following code:
(def env
  (pci/register
   [(pbir/constantly-resolver :products
                              [{:product/id 1 :product/slug "test1"}
                               {:product/id 2 :product/slug "test2"}])]))
(p.eql/process env [{:products [:product/slug]}])
;; => #{:products [#:product{:slug "test1"} #:product{:slug "test2"}]}
(p.eql/process env [{:products [:product/id]}])
;; => #:{:products [#:product{:id 1} #:product{:id 2}]}
(p.eql/process env [{[:product/id 1] [:product/slug]}])
;; => {[:product/id 1] {}}
The first two queries above work as expected, but the last query doesn’t. In order to process
(p.eql/process env [{[:product/id 1] [:product/slug]}])
Do I need to create another resolver to match the product/id ?
#2022-04-2620:58Ī»ustin f(n)Yes, you need a resolver that explicitly knows how to look up a product/slug from a product/id. In your example this could be a resolver that takes :product/id and :products , then filters the products for one that matches.#2022-04-2621:01Ī»ustin f(n)The resolver you have provides :products , but for all pathom knows there is no relation between :product/id and :product/slug (Other than them both being present in :products). They could be completely random data, or magic labels, or whatever.#2022-04-2621:02nharschExcellent, thanks.#2022-04-2700:19nharschDoes anyone have any example projects using Pathom 3 as a client side interface to a Rest Api?#2022-04-2705:08HukkaWell, there's https://github.com/wilkerlucio/presentation-data-navigation-with-pathom3/blob/main/src/main/com/wsscode/presentations/pathom3_data_nav/twitter/v1.clj#2022-04-2705:09HukkaHaven't seen anything complete with a frontend too#2022-04-2720:19wilkerluciothis is how I mainly use Pathom for, do you have any specific questions around this process?#2022-04-3004:20jasonjcknis it possible to autoconvert GraphQL queries into Pathom, for the people in my organization that don’t want to learn datalog syntax.#2022-04-3014:57wilkerlucioit is possible, but not fully automatically, the problems is that the schema on GraphQL must be designed, while every "type" in Pathom is a generic container. one idea I had other day is make a schema on GraphQL that has only one type, and all the attributes (which conceptually matches the Pathom design), the problem in this scenario becomes how to convert back and forth the attribute names (like :foo.bar.baz/thing), GraphQL symbols are much limited when compared to EDN keywords, this ends up needing to make some weird conversion decisions (kinda like what munge does in Clojure), which makes the GraphQL API look terrible#2022-05-0310:47andrewzhurovHi, I'm getting myself familiar with pathom3. I have a couple of resolvers registered that derive such data:
:project/path -> :project/analysis -> :analysis/var-definitions -> :project/vars [:var/ns :var/name :var/expr]
:project/path -> :project/analysis -> :analysis/var-usages      -> :project/vars [:var/ns :var/name {:var/uses-var [:var/ns :var/name]}]
I can't figure out a couple of things: 1. How to have :project/vars normalized by :var/ns + :var/name? 2. How to be able to perform such queries:
[{:project/vars [{:var/uses-var [:var/expr]}]}]
#2022-05-0318:17wilkerluciohello andrew, not clear to me what you intending to do here yet, can you make a running example to point where the misunderstanding may be?#2022-05-0315:35souenzzoHello I'm trying to make the Index Explorer tab at Fulcro Inspector work with pathom3 I tried to make a simple resolver that returns the index:
(pco/defresolver index-explorer [env _]
  {::pco/input  [:com.wsscode.pathom.viz.index-explorer/id]
   ::pco/output [:com.wsscode.pathom.viz.index-explorer/index]}
  {:com.wsscode.pathom.viz.index-explorer/index (select-keys env
                                                  [:com.wsscode.pathom3.connect.indexes/index-resolvers
                                                   :com.wsscode.pathom3.connect.indexes/index-attributes
                                                   :com.wsscode.pathom3.connect.indexes/index-io
                                                   :com.wsscode.pathom3.connect.indexes/index-oir
                                                   :com.wsscode.pathom3.connect.indexes/index-mutations])})
It did not work. Probably because the inspector expect that the index contains a pathom2-style index. There is some buitin function to do this transformationw
#2022-05-0317:24xcenoThere's an old thread in #fulcro about it: https://clojurians.slack.com/archives/C68M60S4F/p1645018882724239 TL;DR: Might have something todo with RAD internals, but no one figured it out yet#2022-05-0318:12souenzzohttps://github.com/souenzzo/pathom-23-compat/#2022-05-0318:15wilkerlucioFulcro Inspect unfortunally uses some old code that doesn't support Pathom 3 yet#2022-05-0318:15wilkerlucioIm glad to help anyone that would like to work on this, because I can't work on it at the moment, but its something I would like to do (or help someone else to do it)#2022-05-0318:16wilkerluciofor start we need to migrate Fulcro Inspect to use Fulcro 3 (because the new versions of Pathom Viz are already in Fulcro 3)#2022-05-0318:16xcenoWell, I forked fulcro-inspect a while ago because I wanted to build a dark-mode and add some virtualization for the dom, but imho it should first be converted to fulcro 3. So in the end I didn't get very far, but I could try again. Will take some weeks though, I'm very busy this month#2022-05-0414:12dvingolol @U012ADU90SW I also wanted a dark theme! I had some time last winter and ported the inspector to f3 and also added a dark theme. I can make a pass this week to clean up what I have and push it out. I've been using it to develop my own apps and seems to work fine.#2022-05-0414:29wilkerlucio@U051V5LLP can you share this with me? would love to merge it back to Fulcro Inspect repo, is there any notes/considerations you think we need to merge it back?#2022-05-0416:10dvingohey @U066U8JQJ sure! - I'll just post the fork here after I clean it up#2022-05-0420:33xceno@U051V5LLP lol no way šŸ˜‚ but that's great! I'm going to have a look at your fork then once you have it ready#2022-05-0615:01dvingogreetings gentlemen, here is what I have: https://github.com/dvingo/fulcro-inspect/tree/dvingo/fulcro3-port I did a pass to remove any silly comments (I hope all of them), and there are a few log statements that you may want to comment out. There's likely a few bugs remaining, I haven't been working on it lately so the memories aren't fresh. Check out the contributing.md for more dev notes, I have run through the instructions there so it should work, but feel free to fork it and have fun!#2022-05-0714:47wilkerlucioawesome! looking forward to check it out#2022-05-0409:20henrikI have a body that can deliver recursive results. I do this by looking at (::pcn/expects (::pcn/node env)) and computing result accordingly. Is it possible to instruct a resolver to expect a recursive return value? I.e., {::pco/output [{:some/thing '...}]} The above doesn't work.#2022-05-0414:24wilkerluciowhat's the goal here? can you give me an example?#2022-05-0512:42henrikSure! So in this instance, I'm passing along queries to XTDB. XTDB is capable of resolving recursive queries itself (it even uses EQL to do so), so rather than issuing multiple requests to XTDB, I could pass along the one request, and receive the full response in one go. That is, it would be possible if I could tell that a recursive query has been requested.#2022-05-0512:43wilkerlucioif I remember right you can see that by looking into the ast, let me do a quick test here#2022-05-0512:52wilkerlucioI believe this is the fragment you are looking for:
(-> env :com.wsscode.pathom3.connect.planner/graph :com.wsscode.pathom3.connect.planner/source-ast :query)
#2022-05-0512:52wilkerlucioif query is ... (the symbol), its a recursion going on#2022-05-0512:52wilkerluciocomplete example:
(ns com.wsscode.pathom3.demos.recursive
  (:require
    [ :as io]
    [com.wsscode.pathom3.connect.indexes :as pci]
    [com.wsscode.pathom3.connect.operation :as pco]
    [com.wsscode.pathom3.interface.eql :as p.eql])
  (:import ( File)))

(pco/defresolver file-resolver
  [{:file/keys [path]}]
  {::pco/output [:file/dir?]}
  (let [f    (io/file path)
        dir? (.isDirectory f)]
    {:file/dir? dir?}))

(pco/defresolver directory-files-resolver
  [env {:file/keys [path dir?]}]
  {::pco/output [{:dir/files [:file/path]}]}
  (tap> (-> env :com.wsscode.pathom3.connect.planner/graph :com.wsscode.pathom3.connect.planner/source-ast :query))
  (if dir?
    {:dir/files
     (mapv (fn [^File f0] {:file/path (.getPath f0)})
       (.listFiles (io/file path)))}))


(def env
  (pci/register
    [file-resolver
     directory-files-resolver]))

(comment
  (p.eql/process env
    {:file/path "src/demos"}
    '[:file/path
     {:dir/files ...}]))
#2022-05-0512:55wilkerlucioif that ends up being a bit unreliable, try using the graph -> index-ast -> the output property, that's for sure the right ast (but requires knowing the attr)#2022-05-0514:18henrikThanks @wilkerlucio! I'll try it out.#2022-05-0414:18dvingoThe query indicates the recursion and then pathom will continue querying based on the results - the resolver should just return a way to map an id to an entity (which may contain more nested ids) This is from my notes on pathom2 (pretty sure I copied it from this slack someplace in the past) but should translate directly to p3:
(pc/defresolver file-resolver [env {:file/keys [path]}]
  {::pc/input  #{:file/path}
   ::pc/output [:file/type]}
  (let [f    ( path)
        dir? (.isDirectory f)]
    {:file/type (if dir? :dir :file)
     :file/dir? dir?}))

(pc/defresolver directory-files-resolver [env {:file/keys [path dir?]}]
  {::pc/input  #{:file/path :file/dir?}
   ::pc/output [{:dir/files [:file/path]}]}
  (if dir?
    {:dir/files
     (mapv (fn [^File f0] {:file/path (.getPath f0)})
       (.listFiles ( path)))}))

 (parser {} [{[:file/path "src"]
               [:file/path
                :file/type
                {:dir/files '...}]}])
#2022-05-0512:47henrikOh yes, I know how to use Pathom recursively. In this case, I have an IO target that is capable of resolving recursion, so I'm looking to optimize it down to one query to the resource, rather than multiple queries as would be issued by Pathom.#2022-05-0514:59dvingoYou can return a tree of data and pathom won't make the recursive resolver calls. I've used this with datalog dbs to issue one a recursive pull query instead of having pathom perform the recursion for a big perf gain#2022-05-0515:16henrikYes, but I don't want to do a big recursive pull unless it's expected. That would produce a lot of overhead for when we don't want it, or know a specific depth we want in advance. Hence, I want to look at what the user expects, and adjust my pull accordingly. This works great with the expects map outside of recursion, but not when recursion is present. My hope was that there would be some way to inform the parser, statically, that we are capable of returning a recursive result, and that this would in turn deliver an expects, that might look something like:
{:some/thing ...}
Rather than,
{:some/thing {}}
This would be handy, since the expects structure can be trivially turned into a pull query structure. Perhaps it would give good static information to Pathom as well, that wouldn't be present if you return a recursive structure as a surprise (rather than a promise). @wilkerlucio has given me some things to check out here, however, so it's not a lost cause yet: https://clojurians.slack.com/archives/C87NB2CFN/p1651656055414599
#2022-05-0515:37dvingoah I see - you can simplify things and pass the query depth as a query parameter#2022-05-0710:32henrikI don't love that idea. The depth is already known, since it is passed in the query. If I require it as an additional parameter on top of the query, I'm burdening the user with technicalities that they shouldn't have to worry about.#2022-05-0713:41dvingoOk, I don't understand what problem you're trying to solve, but seems like you figured it out.#2022-05-0714:46wilkerlucio@U051V5LLP the idea is to keep the same interface working (the ...), the param idea can work but them its not using the eql feature that was designed for that specific purpose, so if we can keep it consistent its better, and that's what @U06B8J0AJ is going for#2022-05-0714:50dvingoI see, so it's fundamentally about a resolver which might be used to resolve both a recursive query and a non-recursive query?#2022-05-0715:34henrikYeah, since I happen to have some IO resources where I can leverage the recursive bit straight inside the resolver, it's an optimization opportunity. The overhead of Pathom becomes effectively nil for queries that stay within the same resource. If I don't do this, then you could argue that calling XTDB directly is more efficient, since that would be 1 call instead of the N calls that Pathom generates for the same pull expression. Now I'll be able to wave aside the naysayers. šŸ™‚#2022-05-0420:05dgr@wilkerlucio, I found this talk the other day and was struck by the similarity to Pathom. While the implementation is not Clojure, it has similar motivations for the project. https://www.hytradboi.com/2022/how-to-query-almost-everything#2022-05-0422:48wilkerlucioyup, conceptually is the same idea, what I've been saying all this time is that by using an attribute driven approach we get a graph system that's much easier to extend over time, without having to keep messing with schemas#2022-05-0423:00dgrRight, exactly. And Pathom is more demand-driven as opposed to explicitly stating which relationships should be traversed in which order, which this seems to do. Anyway, I just found it interesting as a data point that validates both the problem you’re trying to solve as well as the general solution (a graph of data over all the DBs).#2022-05-0512:46Ryan DomiganHi, I'm trying to model a graph using pathom3, and have a transaction which creates multiple nodes and then adds a reference from one to another, something like [(add-node {:id "a"}) (add-node {:id "b"}) (setup-ref {:from "a" :to "b"})]. Is this expected to work? I'm trying to debug and it seems like add-node is running for :id "b" but not "a"#2022-05-0513:45wilkerluciohello Ryan, when you have to coordinate things, its better to pack them in a single operation so you have more control, so instead of 3 mutations, you can design it to be one thing, something like:
[(include-nodes {:nodes [{:id "a"}
                         {:id "b"}]
                 :rels [["a" "b"]]})]
#2022-05-0514:23wilkerlucioalthough, in any case it should be running all the mutations, let me see if there is an issue there#2022-05-0514:25wilkerlucioyeah, this demo shows a bug, multiple mutations are converging to the same params#2022-05-0514:25wilkerlucioI see the problem, will have a fix for it soon#2022-05-0514:27wilkerluciohttps://github.com/wilkerlucio/pathom3/issues/137#2022-05-0514:34Ryan DomiganThanks for letting me know, I'll keep an eye on the issue. I'm populating my DB for testing, and for now I think running separate queries models the expected interaction just as well#2022-05-0514:38wilkerluciothanks from bringing it up, nasty issue that got all this time without being noticed#2022-05-0514:41wilkerluciobut, there is a more fundamental problem to fix in this case, there is what the response will look like? since the key of the mutation is what shows up, they will conflict, we need a way to rename some of them, otherwise we can only get the response for one of them#2022-05-0615:21Ryan DomiganYou can call the same mutation multiple times with the same arguments and get different results, so I think the result needs to encode position of the mutation in the input. Not sure if you were looking for feedback or just thinking out loud.#2022-05-0615:21wilkerluciosure, feedback is always welcome šŸ™‚#2022-05-0615:22wilkerlucioI'm thinking for the first fix, just fix the calling part, and let output a single result (prob the first, or the last one)#2022-05-0615:22wilkerluciothis will fix the calls at least#2022-05-0615:22wilkerluciofor the response, I'm thinkihng of using some special parameter to define the output name, this way the user can be explicit about it#2022-05-0520:06donovanmcgillenHi. Newbie question here, but what is the intended / recommended method for dealing with the situation when the thing a resolver is looking up doesn’t exist at all? E.g. imagine I have a resolver that takes :user/id as an input and outputs :user/id and :user/name. As far as I can tell, there is no difference in the output for a user that exists but doesn’t have a name to a user that doesn’t exist at all - in both cases you get back the :user/id supplied with an attribute missing error on :user/name, but really these are two different things and I want to know the difference. I notice throwing exceptions also adds the exception into each attribute (other than the :user/id) so I think I’m missing something in the philosophy of how to handle this by thinking in terms of attributes. I have been toying with using union queries so I can return either an error or an intended ā€˜thing’ from mutations / resolvers in general when there are errors - perhaps something like that the way to go?#2022-05-0520:37wilkerluciohello Donovan, funny that I was yesterday talking to co-workers just about that šŸ™‚ the philosophical thing here is that each attribute is its own world, the concept of "user exists" or not is not really applicable, not in the same way we are used to when dealing primarily with entities. in Pathom the question is not about "does the user exists?", but "is this attribute available given the context?" this is general way of thinking, but with that in mind, if you have some way to tell if a user exists or not (by hitting a db, service, or whatever) and this information is relevant to your business, then the way in Pathom is to make an attribute that can tell you specifically that, for example an attribute :user/exists? (or even :user/stored-on-db?), and for the implementation of this you can make sure it deals with the difference between non-existent vs don't have a specific attribute, because then :user/exists? becomes something you have full control, and can make any heuristics you want makes sense?#2022-05-0520:42wilkerlucioI dont recommend jumping to unions on this case, unions is more branching for different sub-queries (that may be very different) depending on something in the data#2022-05-0609:31donovanmcgillenHi Wilker, thanks for replying. That makes a lot of sense (and is quite obvious now you mention it!). It's not something I've found I typically need to worry about, I've just hit a case where I have a page of my app that takes the id of a specific thing in it's path (so user's can bookmark) and I want to handle people typing in non-existent ids / ids that don't belong to them as a specific case. One thing I have tried using a union query for (that I'm now wondering is not the way to go based on what you have said) is for getting errors out of mutations. For example, in fulcro I am sending a query along with a mutation (in the remote of the fulcro client side mutation I'm using something like (m/returning env SomeThing) ) and my mutation on success will return something like {:some-thing/id id} that will go on and be resolved, but in some cases I need to return an error from the mutation. What I have been looking at doing is wrapping SomeThing to make a union query to get either the mutation error map or query for SomeThing, but I'm now wondering if perhaps I should let the SomeThing query happen and use the attribute errors - that does mean I lose the reason the mutation failed, though. Another option is just to throw an exception from the mutation (and then I guess use a plugin to turn it into something that can be sent over the wire?). Sorry, not sure if this is a pathom or a fulcro question really!#2022-05-0613:56wilkerluciono worries, these modelling questions are common, since the modeling based on attributes is a new thing for most of the people that start using Pathom, so we are learning together on how to work this way šŸ™‚#2022-05-0613:57wilkerlucioI dont have an immediate answer for the mutation question now, its been a while since I dealt with that in Fulcro, what I remember for sure now is that I used the pessimist mutation in fulcro to handle cases like this, so the error flow is controlled more on the Fulcro end, but of course, Pathom needs to give you something to make that decision there#2022-05-0613:57wilkerlucioI'll have a look into it and get back to you#2022-05-0618:49donovanmcgillenVery much appreciated šŸ‘#2022-05-0714:57dvingohey @wilkerlucio i'm trying to integrate a pathom3 parser/fulcro remote with the latest pathom viz - any pointers on how to augment either my env or which resolvers to add to get things working? I only see the remote version for use with the standalone app should I be looking in this ns? https://github.com/wilkerlucio/pathom-viz-connector/blob/master/src/com/wsscode/pathom/viz/ws_connector/pathom3.cljc I see (pci/register env p.connector/request-snapshots) but I don't think that's everything#2022-05-1012:46wilkerluciothanks, sorry I haven't stopped to check it yet, but its on my todo list#2022-05-1014:22dvingoall good! saw you online the other day and was hoping to catch you šŸ™‚ I'd be interested in bringing over all the pathom-viz views into the inspector. I'll keep digging in in the meantime to see if I can make some progress.#2022-05-1015:35wilkerluciothe recent versions of Pathom Viz already use Fulcro 3, it just need some analisys to get it all together#2022-05-0715:14dvingotrying to convert this view https://github.com/dvingo/fulcro-inspect/blob/dvingo/fulcro3-port/src/ui/fulcro/inspect/ui/index_explorer.cljs is there a new UI component I should be targeting?#2022-05-0715:15dvingo(that view renders https://github.com/wilkerlucio/pathom-viz/blob/master/src/core/com/wsscode/pathom/viz/index_explorer.cljs)#2022-05-0718:31dvingoI see now, I need to pass the map through this transformation pipeline: https://github.com/wilkerlucio/pathom-viz/blob/bf9126438c9fc9cf7d96367cefc0713ef4643fc2/src/core/com/wsscode/pathom/viz/index_explorer.cljs#L1372 I am struggling to make that happen. The relevant data is coming in to the inspector UI here: https://github.com/dvingo/fulcro-inspect/blob/ae370974c9aa599640a07355c3c26ca830550258/src/ui/fulcro/inspect/ui/index_explorer.cljs#L129 and I'm trying this:
(defn clean-indexes [indexes]
  (let [indexes (dissoc indexes
                  :com.wsscode.pathom3.connect.indexes/index-mutations
                  :com.wsscode.pathom3.connect.indexes/index-resolvers)
        indexes (p/elide-special-outputs indexes)
        indexes (p.eql/satisfy adapt/env indexes [:com.wsscode.pathom.connect/indexes])]
    indexes))

(defn clean-explorer [explorer]
  (-> explorer
    (update ::iex/index clean-indexes)
    (update ::iex/idx clean-indexes)))
it's a hack that's not working. anyway, going to take a break. It may be faster for you to install the extension yourself: https://github.com/dvingo/fulcro-inspect
#2022-05-1312:55sheluchinDoes Pathom support field aliases from GraphQL? To make batch queries like this possible, for example:
{
  x: search(query: "sheluchin", type: USER, first: 10) {
    edges {
      node {
        ... on User {
          createdAt
        }
      }
    }
  }
  y: search(query: "sheluchin", type: USER, first: 10) {
    edges {
      node {
        ... on User {
          createdAt
        }
      }
    }
  }
}
#2022-05-1312:57wilkerlucionot supported yet#2022-05-1312:57sheluchinThanks @U066U8JQJ. Would you like an issue for that?#2022-05-1313:06wilkerluciosure#2022-05-1313:06wilkerlucioother than that, the previous fixes solved the things you needed? (just checking :))#2022-05-1313:07sheluchinI'm digging into it right now šŸ™‚ I'll let you know.#2022-05-1313:50sheluchin@U066U8JQJ actually it doesn't look like the fix worked for the first case. Or maybe I'm not doing it right. In any case, when calling the resolver like this the result is the same as reported in the ticket:
(p/let [result (p.eql/process (add-github-gql-to-env)
                    {:github.Organization/login "fulcrologic"}
                    [:github.Organization/all-repos])]
    (tap> result))
But for the second case (GitHub Search) looks fixed.
com.wsscode/pathom3                    {:mvn/version "2022.04.20-alpha"}                          
wilkerlucio/pathom3-graphql            {:git/url "
#2022-05-1313:51sheluchinAnd as before, if I comment this out in the query:
{:github.Repository/owner [:github.RepositoryOwner/id]}
It works.
#2022-05-1313:52wilkerlucio@UPWHQK562 did you bump Pathom 3 as well? you need to use main there#2022-05-1313:52wilkerluciothis commit: 28956c7f5d6dd259effc09567829c096932714a7#2022-05-1313:56sheluchinAh, my bad.. I assumed it was in 04.20. Can confirm it works on that commit! Thank you šŸ™#2022-05-1313:57wilkerluciono worries, its because that bug was a Pathom 3 one related to dynamic query process šŸ˜‰#2022-05-1314:08sheluchinhttps://github.com/wilkerlucio/pathom3-graphql/issues/16 There's the issue for this thread.#2022-05-1611:30roklenarcicI am trying out the error handling in Pathom 3 and I couldn’t get attribute-error function to return a non-nil result. I have a query by ident like [{[:user/id 1] [:user/name]}] and I’ve got the name resolver always throw an Excecption, and lenient mode on. I get what you’d expect:
{[:user/id 1] #:com.wsscode.pathom3.connect.runner{:attribute-errors #:user{:name #:com.wsscode.pathom3.error{:cause :com.wsscode.pathom3.error/node-errors,
                                                                                                                                      :node-error-details ......
But when I call (attribute-error response [:user/id 1]) I get back nil. I’ve looked at implementation of that function and subfunctions and it basicall uses com.wsscode.pathom3.connect.planner/index-attrs and com.wsscode.pathom3.connect.planner/index-ast and com.wsscode.pathom3.connect.runner/node-run-stats . And then I looked at my meta or response and saw that index-attrs was missing, and node-run-stats is empty map…. so of course this didn’t work. What am I doing wrong?
#2022-05-1613:20wilkerluciothe error is always per entity, and attribute error is about an attribute ,an ident query like [:user/id 1] will never have an error on itself because its just a data forward thing, the error will always be inside of it (meaning you need to look for the attribute error at that level), makes sense?#2022-05-1613:22roklenarcicok, so I need to request (attribute-error response :user/name)?#2022-05-1613:22wilkerluciomore like: (attribute-error (get response [:user/id 1]) :user/name)]#2022-05-1613:23roklenarcicthe thing is that node-run-stats is empty and looking at the code it attribute-error it needs to have something in there#2022-05-1613:24wilkerlucionote each map has its own meta with status details#2022-05-1613:24wilkerlucionot just the root#2022-05-1613:24roklenarcicah ok#2022-05-1613:28roklenarcicSo I’ll have to walk the result to log errors I think#2022-05-1613:29wilkerlucioif you want to log errors, its better to use a plugin to wrap error events, its gonna be easier than traversing#2022-05-1613:29roklenarcicYeah I see that#2022-05-1613:30wilkerluciothat said, traversing is possible, and that's what Pathom Viz does to compute the trace tree: https://github.com/wilkerlucio/pathom-viz/blob/master/src/core/com/wsscode/pathom/viz/timeline.cljs#L182-L230#2022-05-1613:31wilkerluciobut for observability, an error plugin is easier and more efficient#2022-05-1614:40roklenarcicLet’s say you’ve got a button, and it’s got ::button/shape resolver, which describes a ā€œtypeā€ (e.g. either circle or a square), and the circle shape has radius, but the square has width, height as attributes, how would you write that resolver output?
(pco/defresolver button-shape [{::button/keys [id]}]
  {::pco/output [{::button/shape [::shape/type ::shape/radius ::shape/width ::shape/height]}]}
  {::button/shape
   (let [b (get-button id)]
     (if (= "circle" (:shape b))
       {::shape/type :circle ::shape/radius (:radius b)}
       {::shape/type :square ::shape/width (:width b) ::shape/height (:height b)}))})
#2022-05-1615:53cjmurphyYou could have a separate resolver for each shape. So a resolver for circle/id and another for square/id. This way Pathom has a resolver (and a <table-name>/id) for each grouping of attributes. Of course I realise that's not answering the question so please take it as a comment.#2022-05-1616:01roklenarcicHow would that work? Union query?#2022-05-1616:07cjmurphyI was suggesting that the schema be changed to accommodate Pathom, so no more types. That's just how I might go about it, given you only have 2 types, and there aren't even any attributes in common. I would have a circle resolver and a square resolver.#2022-05-1619:43wilkerlucioI think your approach is fine, you could go with Unions to make it more specific but I dont think its nescessary (the add complexity of dealing with unions is not worth given the difference in attributes is so short)#2022-05-1619:45roklenarcicBut without unions I have problem where th resolver is in error because it is not returning all the requested properties#2022-05-1621:39cjmurphyWell for those attributes you could return nil or your own key, say :not-needed. That would stop the error. But maybe there's a better way...#2022-05-1622:03wilkerlucioyou can also use optionality to handle the missing data cases, like:
(p.eql/process env [{::button/shape [::shape/type (pco/? ::shape/radius) (pco/? ::shape/width) (pco/? ::shape/height)]}
]
#2022-05-1622:04wilkerlucioor returning nils, as @U0D5RN0S1 mentioned#2022-05-1622:30dehliIs it possible to get whether a :node-id is optional from within the runner? (or does that question not make sense šŸ™‚). I'm looking through the source code and trying to address https://github.com/wilkerlucio/pathom3/issues/138. My thoughts are inside of pcr/run-or-node! after we've gone through all the nodes, if the key is optional, rather than failing fast we could continue w/ execution. Also hi! šŸ‘‹#2022-05-1623:30wilkerluciojust sent a possible fix, let me know if works in your scenario šŸ™‚#2022-05-1623:31wilkerlucioabout the question o optional node, a optional node doesn't make sense, because the node might have both required and optional things to fulfill, the optionality is something we check at the index-ast via the attribute key (looking for the optional param there)#2022-05-1623:31wilkerluciocommit with the candidate fix: https://github.com/wilkerlucio/pathom3/commit/dfa61ad035708c1a001f4ba70743ee1b3c66ded7#2022-05-1623:45dehliThanks! That looks like it would work for my scenario. I'll pull latest for my project. Thanks for answering the node question! That makes sense to me.#2022-05-1623:46wilkerluciocurrently every OR node can have only one attribute expected, but since this is a thing that's not used extensively (this may change in the future with better optimizations), I believe we still gonna find some rough edges, and fix as we find them šŸ™‚#2022-05-1700:15dehliJust confirmed that the fix worked for my more complicated example šŸŽ‰ Thanks for your help and explanation! And I'll happily continue to help surface any rough edges as I find them šŸ™‚#2022-05-1712:50wilkerlucio#2022-05-1909:21JasperHi y'all. Thanks for the new Pathom release wilkerlucio. With the new update all my mutation tests fail when running p.eql/process it says something like: get-node's argument list Spec failed seems to be on :com.wsscode.pathom3.connect.planner/mutations -> should satisfy -> symbol? Does this have something todo with "BREAKING: ::pcp/mutations now contains the full call AST instead of the mutation key". Anyone else seen this or know what I should change?#2022-05-1913:14wilkerluciohello, for regular usage it shouldn't break, its more an internal change (only affecting people that does instrospection on it for some reason)#2022-05-1913:15wilkerlucioah, I see, you are running with guardrails on, right? I must have missed that, this looks like a spec error (error on my end, forget to update the spec)
#2022-05-1913:17wilkerluciochecking it now#2022-05-1913:18JasperYes your libary inspired us to use guardrails in our project. (which is really very helpful on our interfaces, so thank you for that ā¤ļø)#2022-05-1913:18Jasper^ thanks#2022-05-1913:18wilkerluciosorry, I need to get back on running with it, I'm doing the fixes to the bad specs, will be back fine soon šŸ˜‰#2022-05-1913:20Jasperno worries, thanks for doing that#2022-05-1913:45wilkerluciohttps://clojurians.slack.com/archives/C015AL9QYH1/p1652967906959309#2022-05-1913:45wilkerlucioreleased#2022-05-1913:45wilkerlucioplease let me know if things work fine for you with this one#2022-05-1914:03Jasperthanks will test it soon#2022-05-1914:16Jasper@U066U8JQJ its all working now thanks!#2022-05-1913:45wilkerlucio#2022-05-2019:50dehliHi again šŸ‘‹ In com.wsscode.pathom3.connect.runner-test/check-all-runners I’m seeing that the inclusion of extra keys in a runner’s response doesn’t cause the test to fail. Is this by design? Edit: After exploring a bit more, I see that there are additional keys included in runner’s response (such as entity data) so my hunch is it’s for that.#2022-05-2022:38wilkerlucioyeah, it uses matcher combinators library that does a check that allows for extra keys#2022-05-2022:38wilkerluciothere are ways to make strict, but most of the time its ok to let the extra keys#2022-05-2022:39dehlithanks!#2022-05-2022:39wilkerluciothe nice thing is that it also allows for predicates in the middle of the structure, things like: {:uuid (random-uuid)} => {:uuid uuid?} are valid check definitions#2022-05-2022:40dehlioh that is cool! i'll have to check out the library.#2022-05-2022:40wilkerlucioyou can learn more at: https://github.com/nubank/matcher-combinators#2022-05-2022:42wilkerluciohappy to see you taking this dive on Pathom code šŸ™‚#2022-05-2022:43wilkerlucioI like to use this though another library called check from @U3Y18N0UC: https://gitlab.com/mauricioszabo/check#2022-05-2022:46dehliawesome! will check that out as well. it's fun to see a little more of how all the magic happens with pathom šŸ˜„#2022-05-2418:56sheluchinI've read some people describe using Pathom as a tool for generic function composition. Are there any articles that elaborate on this way of thinking with some examples?#2022-05-2419:03markaddlemanI've been thinking of writing an article on this. I think of Pathom as automatic function composition.#2022-05-2420:47souenzzoI did this a few days ago. Not an article, but related. https://github.com/souenzzo/sciphi#2022-05-2421:48sheluchin@U2845S9KL It may have been something you mentioned that got me thinking about it. I've been trying to find more places to use Pathom in my code, but I feel like I'm not really getting the full potential. I don't have a clear picture in my head about how it should work when, as you described, you have several resolvers that get attempted until one returns something other than nil. I mean, I can describe it, but I don't see how I'd re-implement my existing code that way to get value out of it. As for the concept of going up one level of abstraction and returning things like object expressions rather than the objects -- that's hard to wrap my head around at all. And further, I find myself shying away from using Pathom where a regular function would be "good enough" because I'm worried about adding unnecessary complexity or potential latency, but I think this sort of thinking is what's holding me back from understanding "Pathom as automatic function composition". I recently read Out of the Tarpit and thought the parts about artificial execution order requirements creating unnecessary complexity were relevant to Pathom's value statement about "abstracting the function call chain"... so here I am trying to make more sense of how to leverage it. Hopefully that gives you some more ideas for that article šŸ™‚#2022-05-2421:50markaddlemanIn my current project, I have structured the entire backend data model around Pathom not just the client facing API#2022-05-2421:51markaddlemanThere are definitely tradeoffs. On the positive side, it forces me to be more thoughtful about the data model and that has lead to better structure.#2022-05-2421:52markaddlemanBut the downside is that data model is harder to navigate because my tooling doesn't understand it like it understands function calls#2022-05-2421:53markaddlemanSome of th navigation problem is ameliorated by Pathom Viz but that tool isn't quite where it needs to be (yet)#2022-05-2421:54markaddlemanYou have definitely given me more reason to flesh out my thinking in an article. Now I just need to find the time 😃#2022-05-2421:54sheluchin> But the downside is that data model is harder to navigate because my tooling doesn't understand it like it understands function calls Ah, yes, I don't know how many times I've thought "it would sure be nice if the code lens could give me reference counts on these attributes".#2022-05-2421:55markaddlemanOverall, I'm happy with the decision. In fact, I'm thinking of writing a macro that blurs the line between a resolver and regular function even more.#2022-05-2421:56markaddlemanThere are some cases where my application needs parameters that are computed by a client facing resolver AND would benefit by pulling additional information out of the Pathom environment#2022-05-2421:58markaddlemanI'd like to invoke that code as a regular function (out of convenience) by passing in both the environment and a list of parameters rather than bind it all up in an entity map.#2022-05-2421:59markaddlemanI just looked at the link @U2J4FRT2T posted . That might be what I'm talking about#2022-05-2514:44souenzzoThe current sciphi impl only resolve keys in function calls. Today I think that it could do something directly on maps. I will try to do it#2022-05-2615:02wilkerluciohello @UPWHQK562, about the mental model, there a few things that I like to keep in mind that help me, and may help you too: 1. everything is a graph: the first big mental thing here is to imagine that all your data is a single graph, where nodes may contain data and/or links to other nodes (entities), in abstract that's possible any data because we deal and think with data that has those relations/connections 2. generic entities: this is the idea where every node in this graph is generic (in contrast with a GraphQL graph for instance, where the nodes must have specific types), and what the node is/can do depends on the data available at it, that when combined with resolvers will define the reach of that node (both in terms of the data on itself, and the relationships of it) them its about defining these atributes and mapping them šŸ™‚#2022-05-2615:06wilkerlucioI find souenze idea a very interesting experiment, in practical terms I wouldn't go that further to make everything on Pathom, mainly for 2 reasons: 1. Pathom is not performatic enough to replace standard call stacks, if you need real speed, better to compute "out of the graph" and just put it back under a name. 2. not everything needs a name, Pathom is more on the controller side, but depending on the complexity of the process, its easier/better/faster to dont have the names for every intermediate step, the names that make sense are the ones you plan to use somehow, and the good thing is that you can always add names in the middle if you need later without affecting the users, this frees you to decide later what intermediate steps you want to name#2022-05-2714:41sheluchinThanks for the input everyone. I think I'll continue trying to use Pathom in more places, but will look out for the performance hiccups. I'm still not totally clear on how it works as automatic functional composition, but hopefully I'll build up that intuition with more experience:crossed_fingers:#2022-05-2715:35markaddlemanI'd love to hear what you discover on your journey (yeah, I know I need to write up my thoughts as well šŸ™‚ ).#2022-05-2715:39markaddlemanIn response to @U066U8JQJ’s comment, "not everything needs a name" - This is true but I find that Pathom doesn't help or hinder in this regard. Without Pathom, I have to name the function. It's true that Pathom makes me come up with a few different names (attribute names and the resolver name), this isn't a big deal. In fact, the structure that that Pathom enforces helps me regularize names so that names can better serve their purpose: helping me keep the application straight in my head.#2022-05-2715:40markaddlemanThis isn't to say that I think resolvers should be used everywhere that functions are used. I definitely do not. I haven't figured out where that boundary is yet, though.#2022-05-2721:26Bjƶrn EbbinghausSome days ago I drew an activity diagram and told a colleague that I would love to have some visual programming tool, that could translate something like this into code, parallelized it, if possible… Pathom does effectively that…#2022-05-2721:29Bjƶrn EbbinghausIn my case, my functions are either throwing if something goes wrong or returning a vector which gets merged with other vectors into a datahike transaction..#2022-05-2914:12sheluchin@U4VT24ZM3 that's the pattern you use throughout your https://github.com/hhucn/decide3/blob/7c048b0411285282d56a127cd1ec10362d024947/src/main/decide/models/argumentation/api.clj#L68= project? That looks like a nice set of examples for this topic.#2022-05-2920:49Bjƶrn EbbinghausHm? The "Return vectors and concat them into a single transaction" pattern?#2022-05-2921:51sheluchin@U4VT24ZM3 Yes, I was just looking at your code to see if I could find some concrete examples. Think I need to take a deeper look to understand it better, but some of the resolvers there look like what you were describing. Am I mistaken here?#2022-05-2922:21Bjƶrn EbbinghausI donā€˜t use resolver for this right now. The resolvers are purely for the Fulcro frontend. (Be careful with the code. Some parts are quite old and hastily cobbled together for an experiment. The software is used for my research.) Functions which produce vectors for transaction are prefixed with -> #2022-06-0219:33JAtkinsThis doesn't exist in the public domain yet, but a project I'm working on uses pathom as the extension interface. i.e. plugins can install their own compute-able subgraphs to the main app. While large scale plugins haven't been made (yet), I'm quite happy with the composition it gives me in the small scale.#2022-06-0314:08markaddlemanfwiw, we have something similar. We use a combination of pco/priority and when clauses to support something like multimethods in Pathom. I think the approach is more powerful than multimethods since we don't have to decide on a dispatching function up front. The only thing that we need to be concerned about is the order in which resolvers fire which is very easy to reason about (now that resolvers are executed strictly using priority).#2022-05-2513:13Matt SecoskeI’m not seeing a way to get a request context in a pathom3 mutation… am I missing something?#2022-05-2513:48nivekuilthe env? it's an optional arg now#2022-05-2518:07Matt SecoskeGotcha, thanks!#2022-05-2518:42wilkerluciojust to be clear, the macro pco/defmutation makes it optional, when using pco/mutation for example you need to use 2 args always (same as pco/resolver vs pco/defresolver)#2022-05-2716:10Bjƶrn Ebbinghaus@wilkerlucio Finally looking into migrating my app to pathom3 Playing around with Pathom Viz I noticed something interesting. I have the following query with a placeholder.
[{[:decide.models.process/slug "wie-digitalisieren-wir-die-hhu"]
  [{:decide.models.process/proposals
    [:decide.models.proposal/no-of-arguments]}
   {:>/favorite-list
    [{:decide.models.process/proposals
      [:decide.models.proposal/no-of-arguments]}]}]}]
I expected for the graph with the placeholder to just be "merged" into the parent query and then spread out again later, without any real overhead… But this doesn't seem to be the case? Resolvers are still called, but are cached? Is this correct behaviour? Is there a way to optimize this?
#2022-05-2716:15Bjƶrn EbbinghausFrom this query:
[{[:decide.models.process/slug "wie-digitalisieren-wir-die-hhu"]
  [{:decide.models.process/proposals
    [:decide.models.proposal/nice-id
     {:>/foo 
      [:decide.models.proposal/body]}
     {:>/bar
      [:decide.models.proposal/title]}]}]}]
With a resolver like:
(defresolver resolve-proposal ...
  {::pc/output [::proposal/id ::proposal/body ::proposal/title]}
  {::proposal/id 1 
   ::proposal/body "Body" 
   ::proposal/title "title"})
I expected that the corresponding resolver is only called once and the result is shaped correspondingly later.
#2022-05-2716:22Bjƶrn EbbinghausBut the trace for this is really long. Sure. The follow-up calls to the resolver are fast, because the resolver before already provided the values, but still. I can't stop thinking that this is something a planner could optimize.#2022-05-2716:27Bjƶrn EbbinghausHm… Looking at the snapshot view, the planer already optimizes this by merging sibling resolver calls. Then What am I seeing here?#2022-05-2716:38Bjƶrn EbbinghausEvery attribute is resolved by the same resolver:
[{[:decide.models.process/slug "wie-digitalisieren-wir-die-hhu"]
  [{:decide.models.process/proposals
    [:decide.models.proposal/nice-id
     {:>/foo 
      [:decide.models.proposal/body]}
     {:>/bar
      [:decide.models.proposal/title]}
     {:>/baz
      [:decide.models.proposal/created]}
     {:>/bla
      [:decide.models.proposal/original-author]}]}]}]
Takes ~220ms
[{[:decide.models.process/slug "wie-digitalisieren-wir-die-hhu"]
  [{:decide.models.process/proposals
    [:decide.models.proposal/nice-id
     :decide.models.proposal/body
     :decide.models.proposal/title
     :decide.models.proposal/created
     :decide.models.proposal/original-author]}]}]
Takes ~100ms
#2022-05-2716:45wilkerluciohello Bjorn, that seems a lot of extra overhead, do you have guardrails on?#2022-05-2716:45Bjƶrn EbbinghausHm... Yes...#2022-05-2716:45wilkerluciotry turning it off, makes a ton of difference for these small processes šŸ™‚#2022-05-2716:46Bjƶrn EbbinghausThat was a thing I also wanted to look into... Is there a way to disable Guardrails for certain namespaces?#2022-05-2716:46wilkerluciobut talking more about how placeholders work, its not just a merge, that's because the "merged" part is just the first depth layer, placeholders might need further process to run each item, that's why they still need to be processed (and not just merged)#2022-05-2716:47wilkerlucioabout the localization, I remember talking to @U0CKQ19AQ in the past about it, but dont remember if was implemented#2022-05-2716:49Bjƶrn EbbinghausBut isn't discerning how much processing a node needs the job of the planner? I do not quite understand what you mean with "further process"? Do you mean things like running a resolver plugin or something?#2022-05-2716:50wilkerlucioI mean nested requirements, for example:
(comment
  (p.eql/process env
    [{:>/a [:user/a]}
     {:>/b [:user/b]}
     {:>/c [{:user/c
             [{:with-deep 
               [:nested
                {:things [:here]}]}]}]}
     {:>/d [:user/d]}
     {:>/e [:user/e]}]))
#2022-05-2716:52wilkerlucioI can take a deeper look, its been a while since I touched this part of the code, that's what I'm remembering now about it, without guardrails the overhead is much less problematic#2022-05-2716:52Bjƶrn EbbinghausBut this is something the planner can know beforehand, doesn't it? What may be the difference to:
[:user/a
 :user/b
 {:user/c
  [{:with-deep
    [:nested
     {:things [:here]}]}]}
 :user/d
 :user/e]
?
#2022-05-2716:53wilkerlucioit does, but the planner only looks in depths up to a point, it does check it to make sure it will be possible to run (stopping early if it sees its impossible according to the indexes), but each depth level gets its own planning, because the plan depends on the output of the nested item (which may or may not fulfill the complete list of what it exposes in the output, and in the case it doesn't, the plan may turn out different)#2022-05-2716:55wilkerlucioso any time you have to process an entity down, pathom will calculate the plan again, which in most cases shouldn't be a problem, specially if you use a persistent cache for the planner (which is always recommended to do in prod environments)#2022-05-2716:59Bjƶrn EbbinghausWithout guardrails, the overhead is indeed smaller, but still a +10-20%#2022-05-2717:00Bjƶrn EbbinghausEven if the attribute is the same:
[{[:decide.models.process/slug "wie-digitalisieren-wir-die-hhu"]
  [{:decide.models.process/proposals
    [:decide.models.proposal/body
     {:>/foo 
      [:decide.models.proposal/body]}
     {:>/bar
      [:decide.models.proposal/body]}
     {:>/baz
      [:decide.models.proposal/body]}
     {:>/bla
      [:decide.models.proposal/body]}]}]}]
It is the same overhead.
#2022-05-2717:03wilkerlucioyeah, that's correct, I would be happy to make it better, but at this moment I dont see any easy ways to improve that, this 1ms extra per entity gets harder to make right without risking the correctness of the result#2022-05-2717:04Bjƶrn EbbinghausWhat I don't understand is what the snaphot log of the planner is telling me then… It optimizes by "merging the sibling nodes together"… What does that mean if the resolver is still called x times..?#2022-05-2717:05wilkerluciothis is the plan for a single entity, its not that it has 5 calls on the same plan, the plan optimizes and have 1, but you have 5 different plans (one for each placeholder, plus the parent one)#2022-05-2717:06wilkerluciowhen looking from the trace, each plan must have a single node in this case#2022-05-2717:07wilkerluciothe snapshots is telling you the process to build the plan, in this case it does merge all the placeholders to a single graph to try to optiize the deps between then#2022-05-2717:07wilkerlucioaltough, thinking here, in this case the plan for the children should be empty maybe? I gotta double check on that#2022-05-2717:09Bjƶrn EbbinghausAnother thought that I had in the past: It would be great if placeholders could be a concern of the client, wouldn't it? It would reduce redundancy in the query and the result. But I suppose transit would deduplicate that anyway?#2022-05-2717:09wilkerlucioah, reading my code here, the reason it doesn't move the data down is because there may be different parameters at different placeholder levels, and this case we can't re-use the data from the parent#2022-05-2717:10Bjƶrn EbbinghausBut the planner knows that there aren't any parameters, does it not?#2022-05-2717:10wilkerluciothere is an optimization opportunity here, it needs to ensure that the placeholder doesn't have any different parameters in the query when compared to the parent, in this case it can forward the data down#2022-05-2717:15wilkerlucioI just did a test here, and making this optimization doesn't seem to improve the perf a lot#2022-05-2717:15wilkerlucioit may in some more complicated cases, but also requires adding the cost of the param check#2022-05-2717:30Bjƶrn EbbinghausYou may be right. I was shocked surprised at first that the placeholder added +50-100% overhead. But with guardrails disabled, it is much more reasonable. Like a +7% overhead.#2022-05-2718:05wilkerluciowith guardrails it has to keep checking the env in a lot of points, so does the insane overhead šŸ˜›#2022-05-2718:05wilkerlucioI'm interested if you see that is being a real issue, so far it seems to be fine, but I'm open to learn if you find a case where the overhead is prohibitable#2022-05-2718:06wilkerluciobecause it really depends on what else is going on, in general there is a lot of IO that eclipses these smaller processing overheads, but if every resolver is super fast, than it starts to show up more#2022-05-2718:44Bjƶrn EbbinghausI will definitely come back to you if I notice something interesting. btw. Is there a way to get the current nodes "expected output" as EQL? For a resolver that is practically just a d/pull?#2022-05-2718:45wilkerlucioyes#2022-05-2718:45wilkerluciolook for (-> env ::pcp/node ::pcp/expects)#2022-05-2718:46wilkerlucioit uses the shape descriptor format, there are helpers to transform from that to EQL in the ns com.wsscode.pathom3.format.shape-descriptor, the fn shape-descriptor->query#2022-05-2718:47Bjƶrn EbbinghausAh. Yes thank you.#2022-05-2719:04wilkerlucio@U4VT24ZM3 have you tried to use guardrails async? maybe that will help with the overhead#2022-05-2719:06Bjƶrn EbbinghausI used guardrails async:
{:defn-macro nil
 :expound {:show-valid-values? true
           :print-specs? true
           :theme :figwheel-theme}
 :async? true
 :throw? false
 :emit-spec? true}
#2022-05-2719:11wilkerluciowhat emit-spec? does?#2022-05-2719:17Bjƶrn EbbinghausI think it is not used anymore. In the past :emit-spec? false would cause >defn to not emit a fdef spec.#2022-05-2720:16Bjƶrn Ebbinghaus@wilkerlucio In the following example, you could optimize the second AND away, couldn't you?#2022-05-2720:17Bjƶrn EbbinghausIn which case, you could optimize the second resolve-proposal away as well.#2022-05-2720:17wilkerlucioit could, just nothing that's been looked up at this point, the current state is more like doing the optimizations that are safe, its more about finding those patterns and adding more optimizations#2022-05-2720:17wilkerluciothe hardest ones are around OR nodes, those get very few optimizations at this point#2022-05-2720:18wilkerluciobut making those generic is not a trivial job, so they have to be made with a lot of care to avoid messing the execution#2022-05-2720:18wilkerlucioso currently it just lets those happen, also because their cost isn't high, given that it will skip the repeated nodes at execution time#2022-05-2720:19wilkerlucio(but its a problem if you are making them dynamic, like using the expects thing)#2022-05-2720:21wilkerlucio(this approach is due to my experience with the first version, where I was trying to get optimizations at max, to them realize it did a lot of plannings that were just wrong, this made me more cautious about them)#2022-05-2720:23wilkerlucioaltough, to fix the problem with dynamic calls, there is something you can do on your end, you can look at the graph and find all the instances of the same node type, merge their expects and use that to make the call#2022-05-2720:24Bjƶrn EbbinghausBut shouldn't merging ANDs always be possible?#2022-05-2720:25wilkerlucionot always, because they may have follow-up calls with different sets of dependencies, in this case you sent its clear, but sometimes the AND node itself has some nodes after it, or the instances may also have follow up calls, and merging those is trickier#2022-05-2720:26wilkerluciobut this could be an interesting one, in a case where none of the AND nodes have a follow up node (black arrows), then they are mergeable#2022-05-2720:28wilkerluciothese kinds of changes can also affect the paralellism when doing parallel processing, and merging them might result in slower processes (because in this case the merge will make the AND node requires more things before calling the follow up node, delaying that call)#2022-05-2720:28Bjƶrn EbbinghausCan you provide an example, where this doesn't work? In my understanding, an AND node would never have a follow-up node, wouldn't it?#2022-05-2720:28wilkerlucioit does, when you have a node that depends on multiple attributes#2022-05-2720:31wilkerlucio#2022-05-2720:31wilkerlucio
(pco/defresolver multi-dep [{:keys [a b c]}]
  {:d "val"})

(defonce plan-cache (atom {}))

(def env
  (-> {:com.wsscode.pathom3.connect.planner/plan-cache* plan-cache}
      (pci/register
        [(pbir/constantly-resolver :a 1)
         (pbir/constantly-resolver :b 2)
         (pbir/constantly-resolver :c 3)
         multi-dep])
      ((requiring-resolve 'com.wsscode.pathom.viz.ws-connector.pathom3/connect-env)
       "debug")))

(comment
  (p.eql/process env
    [:d]))
#2022-05-2720:32wilkerlucioone a bit more complex#2022-05-2720:32wilkerlucio
(pco/defresolver multi-dep [{:keys [a b c]}]
  {:d "val"})

(pco/defresolver multi-dep2 [{:keys [d e]}]
  {:f "val"})

(defonce plan-cache (atom {}))

(def env
  (-> {:com.wsscode.pathom3.connect.planner/plan-cache* plan-cache}
      (pci/register
        [(pbir/constantly-resolver :a 1)
         (pbir/constantly-resolver :b 2)
         (pbir/constantly-resolver :c 3)
         (pbir/constantly-resolver :e 5)
         multi-dep
         multi-dep2])
      ((requiring-resolve 'com.wsscode.pathom.viz.ws-connector.pathom3/connect-env)
       "debug")))

(comment
  (p.eql/process env
    [:f]))
#2022-05-2720:33wilkerluciothose 2 and nodes can't be merged (the last example)#2022-05-2720:36wilkerluciobut as I said, I think if the inner AND doesn't have a next node, them it should be fine to merge#2022-05-2720:37wilkerluciocan you send me that example that made your graph? I can take a look in optimizing that scenario (just not the OR part)#2022-05-2720:38Bjƶrn EbbinghausThis feels weird to me... I would have expected something like:#2022-05-2720:39wilkerluciothere is a kind of inversion in the order with the run next, in the case of AND its like: get all the deps, THEM run this#2022-05-2720:39wilkerlucioorange arrows go first (branch nodes) and black go after them (run next)#2022-05-2720:42wilkerluciomakes sense?#2022-05-2720:45sheluchin> Is there a way to disable Guardrails for certain namespaces? There was this related issue but I don't think it was implemented https://github.com/fulcrologic/guardrails/issues/26#2022-05-2720:45Bjƶrn EbbinghausHm… I think I am beginning to understand … I thought of it as a dependency graph, but it is more like a "topological layer graph"?#2022-05-2720:46wilkerlucioI see it as an execution graph, but yeah, the point is to optimize for the runner, since the plans can be cached the performance of running depends on how easy it is to traverse#2022-05-2720:46wilkerlucioso what I had in mind is to try to make it as simple as possible, in this case, the runner starts at the root, and there is code for each type of node (resolver, AND or OR nodes)#2022-05-2720:46Bjƶrn EbbinghausYes. execution graph is the right term#2022-05-2720:48wilkerlucioone nice property of this, for the parallel runner, is that it can always parallelize the branches of every AND node#2022-05-2720:51Bjƶrn EbbinghausIn your example it could parallelize a, b, c, e together.#2022-05-2720:51wilkerlucioexactly#2022-05-2720:52wilkerlucioalso, it only waits for a, b and c to start realizing d, maybe it realizes d even before e is done#2022-05-2720:52Bjƶrn EbbinghausAh.. Sure.. Orange arrows can run in parallel...#2022-05-2720:55wilkerlucioin case you are going to try the parallel runner, remember it needs async things to be able to paralellize, otherwise it still goes serial, this is actually good, because it can retain some of the speed of serial for things that are fast#2022-05-2720:55wilkerlucio(in other words, to give Pathom a chance to run things in parallel, a resolver must return a promise, otherwise Pathom will still wait for that resolver to be done before moving forward)#2022-05-2720:56Bjƶrn EbbinghausBut wouldn't that result in "too-much processing" in parallel? I mean, how do you parallelize an OR node?#2022-05-2720:56Bjƶrn EbbinghausOr in this case resolve-proposal :#2022-05-2720:57wilkerlucioit doesn't at this point, it tries each one serially, but, it could be possible to have a different strategy for the OR, it could run all in parallel and pick the first that comes back)#2022-05-2720:57wilkerluciowhat strategy works better will depend on the usage case#2022-05-2721:10Bjƶrn Ebbinghaus
(def graph
  {:a #{}
   :b #{}
   :c #{}
   :d #{:a :b :c}
   :e #{}
   :f #{:e :d}})

(defn topological-layering [graph]
  (loop [graph  graph
         layers []]
    (if (empty? graph)
      layers
      (let [no-deps-nodes         (map key (filter (comp empty? val) graph))
            no-deps-removed-graph (apply dissoc graph no-deps-nodes)
            new-graph             (update-vals no-deps-removed-graph #(apply disj % no-deps-nodes))]
        (recur new-graph (conj layers no-deps-nodes))))))

(topological-layering graph)

; => [(:a :b :c :e) (:d) (:f)]
#2022-05-2721:12Bjƶrn EbbinghausHm… Maybe this is too naive…#2022-05-2721:29wilkerlucioif you like to learn it more in depth I invite you to take a read in the planner namespace, everything related to planning lives there#2022-05-2721:30wilkerluciothe current approach is to first "expand the graph", doing it simply, and optimizing after#2022-05-2721:30wilkerluciothere you will see things like considering optionals, nested input requirements, dynamic resolvers... lots of fun XD#2022-05-2721:31Bjƶrn EbbinghausHave you considered sending that problem to advent of code or some competitive programming event? šŸ˜…#2022-05-2721:33wilkerlucioI find quite hard to explain the problem in detail, haha#2022-05-2721:35Bjƶrn EbbinghausWell, the overall problem is nothing new and "easy" (still NP-hard I think) But you have to fight a lot of nuances as well …#2022-05-2721:37Bjƶrn EbbinghausMaybe I can convince my colleges from the chair for operating systems to give that problem to some students looking for master thesis topic …#2022-05-2721:48wilkerluciohaha, awesome, I would be down for a meeting to give more details on the context and requirements of it#2022-05-2723:01Bjƶrn EbbinghausThis is a brain melter… I should really go to sleep…#2022-05-2914:15wilkerlucioforgot to mention, the planner docs page can also help understand: https://pathom3.wsscode.com/docs/planner#2022-05-2921:18Bjƶrn EbbinghausThanks, I read it Friday night, but I had to get away from it, before it consumed my whole weekend. šŸ˜„#2022-05-2921:51wilkerluciothat doc still a bit superficial imo, but its good for the basics of it#2022-05-2921:51wilkerluciofeedback very welcome to improve it#2022-05-2922:13Bjƶrn EbbinghausNot consumed by reading the docs, but by trying to ā€žsolve the problemā€œ. I am bad at pulling myself away from an interesting ā€žpuzzleā€œ. #2022-05-2922:19wilkerluciohehe, so fun right?! I think the pathom planner is the most complex thing I have done in my life as a programmer, even though I have spend dozens of hours on it, when I need to come back to debug I still have to spend a good time to remember how a part of it works#2022-05-2720:24lgesslerhi, I find myself wanting to write a thin HTTP wrapper around a pathom parser so that consumers can have a familiar HTTP API instead of having to learn pathom, is anyone aware of any reference impls of something like this?#2022-05-2808:26Bjƶrn EbbinghausDo you mean REST API?#2022-05-2816:06lgesslerwell, suppose it could be RESTful, but said HTTP since I (still) don't understand what REST compliance is 🄲#2022-05-2816:09lgesslerI just mean an HTTP API that could have, say, one endpoint per mutation, and one endpoint for every distinct pathom query you want to expose#2022-05-2816:10lgesslerI think it's very obvious what to do here at a basic level, just wanted to check if there are any existing patterns for it to observe#2022-05-3010:05HukkaConsidering that the graph queries could have infinite variation (if there is any recursion), then nothing automatic would work. So it would be pretty much just writing every endpoint by hand, but that doesn't leave lots of code.#2022-05-3010:06HukkaBut yeah, I suppose some kind of automation could work with constraints. Something like no nesting, every resolver gets an endpoint, and the generator makes a way to choose which parts of output is included, plus any possible parameters#2022-05-3010:06HukkaIt just isn't quite as powerful as making custom decisions for what level of nesting is useful and reasonable#2022-05-3022:01wilkerlucioone way I think its interesting to approach this, is to think of the definition of each HTTP endpoint to be just about capturing inputs (from query params, path params, etc...) and building a query, this should be easy enough to abstract away, then your endpoint definitions would look something like this:
(defn user-by-id-handler [{:keys [pathom-boundary-interface] :as request}]
  (pathom-boundary-interface
    {:pathom/entity {:user/id (-> request :path-params :user-id)}
     :pathom/eql    [:user/name
                     :user/email
                     {:user/addresses
                      [:address/street]}]}))
#2022-05-3121:43lgesslerthanks all for the ideas--yeah, i think it's sufficient in my case at least to just assume the query will be static for a given endpoint, which avoids some tricky problems#2022-05-3121:45lgesslereven though consumers of this HTTP API won't have access to pathom's full expressivity it's good to know that maintenance of the API should be very straightforward (just a matter of editing the EQL query and param-harvesting code)#2022-05-2720:47nivekuilhas anyone thought about supervision with pathom's runners? For example the parallel runner doesn't seem to cancel all running promises after one throws#2022-05-2720:48wilkerlucionot yet, but I'm interested in the topic, I think there is still a lot to improve in the parallel runner#2022-05-2720:50wilkerlucioand you right, no cancelling happens at this point, but I think wouldn't be too hard to make that work, we could use some atom at the env with a boolean that starts off, and if we want to stop the whole thing, set it to true, and them the node runner code could halt if it sees it true#2022-05-2720:51wilkerluciothe cancelation thing of things already running might be a bit trickier, we would need generic support at the promises level to allow us to request a cancel in the currently running operations#2022-05-2720:51wilkerluciofor now, if you like to experiment, I think you can write a plugin on wrap-resolve that does with I just described#2022-05-2720:56nivekuilI've been using missionary and pathom together for this in one project#2022-05-3014:04Bjƶrn EbbinghausHow do I update the env with pathom3 in a mutation? In pathom2 you just returned ::p/env#2022-05-3014:09Bjƶrn EbbinghausHmm… I used that functionality to update the :db key in my env to :db-after from the transaction report. https://clojurians.slack.com/archives/C87NB2CFN/p1626499102252200?thread_ts=1626471981.239700&amp;cid=C87NB2CFN#2022-05-3015:02Bjƶrn Ebbinghaus@U066U8JQJ What would be the right place to put things like the current database value or user session or something…? Currently, I am satisfied with something like this:
(defresolver resolve-user-preferences [{:keys [current-user] :as env} _]
  {::pco/inputs []
   ::pco/output [:user/preferences]}
  {:user/preferences (get-preferences env current-user)}) 

(defmutation register-user [env user]
  {::pco/output [:user/id]}
  (let [id (register-user! env user)]
    {::p/env (assoc env :current-user user)
     :user/id id}))
#2022-05-3021:54wilkerlucioI removed that env feature because it was accidently leaking stuff sometimes, currently I think an option is to have some atom at env that you can modify, if path matters this atom could be something like {[:path] :some-value}, this way when reading from the env you use the current path to differenciate branches, makes sense? I'm also open to suggestions, just didn't put much though on this problem at Pathom 3 yet#2022-05-3110:18Bjƶrn EbbinghausDo mean leaking outside the subgraph? In which situation does this occur? In this case, the atom would leak as well, so you gained nothing over adding the value with the path to the env.#2022-05-3117:56wilkerlucioI mean the whole env leaking out in the output data#2022-05-3016:50preporHi. I'm trying to figure out how pathom works with nested data and "contexts". I think it would be easier to just show example/reproducer:
(pco/defresolver situations [{:keys [module/id]}]
    {::pco/output
     [{:module/situations [:situation/id]}]}
    {:module/situations [{:situation/id :s1}
                         {:situation/id :s2}
                         {:situation/id :s3}]})

  (pco/defresolver situation-done [{:keys [situation/id]}]
    {::pco/output
     [:situation/done]}
    {:situation/done true})

  (def env
    (pci/register [situations
                   situation-done]))

  (comment
    (p.eql/process
     env
     {:module/id :m1}
     [{:module/situations [:situation/done]}])
    )
=> #:module{:situations [#:situation{:done true} #:situation{:done true} #:situation{:done true}]}
It works as expected. Now I want to add an input to situation-done resolver, something completely disconnected from situation/id. And I provide it as initial data.
(pco/defresolver situation-done [{:keys [situation/id user/id]}]
  {::pco/output
   [:situation/done]}
  {:situation/done true})

(p.eql/process
   env
   {:module/id :m1
    :user/id 1}
   [{:module/situations [:situation/done]}])
It stops working and says that
Graph execution failed: Pathom can't find a path for the following elements
   in the query: [:situation/done] at path [:module/situations 0]
Why? Isn't user/id resolvable in any "context"? Thank you!
#2022-05-3017:28Bjƶrn Ebbinghaus:user/id is only visible at the "root node" The situations are a node down from the root. Your Graph effectively looks like this:
{:module/id :m1
 :user/id 1
 :module/situations [{:situation/id :s1}]}
Having the :user/id on every level would break things Imagine following situation:
(defresolver resolve-friends [{:keys [user/id]}]
  {::pco/output [{:user/friends [:user/id]}]})

(defresolver resolve-user-name [{:keys [user/id]}]
  {::pco/output [:user/name]})

(process env {:user/id 42} [{:user/friends [:user/id]}])
#2022-05-3017:29Bjƶrn EbbinghausIf you want something available everywhere use env#2022-05-3018:05preporThank you. So, I made wrong assumptions about what pathom (it's funny I made it, but it's out of topic :) ) Could you suggest how to solve situations when I want to keep some context about required data and use it in resolvers? For example, imagine this data structure:
{:room/id :room1
 :room/users [{:user/id :user1
               :user/name-in-room "User name"}]}
To resolve :user/name-in-room I need two things: room/id and user/id. The only way that I see now it's providing it together with :room/users list, but it reduces composability and, possibly, performance. Not every query could want to get :user/name-in-room but it requires additional requests to get this data.
#2022-05-3020:22Bjƶrn EbbinghausMaybe:
{:room/id :room1
 :room/users [#:room+user{:name "User name"
                          :user {:user/id :user1}
                          :room {:room/id :room1}}]}
?
(defresolver resolve-room->room+user [{:keys [room/id]}]
  {::pco/output [{:room/users [:room+user/user :room+user/room]}]})

(defresolver resolve-user->room+user [{:keys [room/id]}]
  {::pco/output [{:user/rooms [:room+user/user :room+user/room]}]})

(defresolver resolve-room+user-name [{:room+user/keys [user room]}]
  {::pco/output [:room+user/name]})
#2022-05-3021:56wilkerlucioI have used this pattern many times, where you forward some data down, so you have the full context you need, something like:
{:room/id :room1
 :room/users [{:user/name "User name"
                          :user/id :user1
                          :room/id :room1}]}
#2022-05-3107:40Linus EricssonYou cannot destruct both user/id and subscription/id because both of them goes to the symbol id#2022-05-3108:39prepor@UQY3M3F6D you are right, but the problem not in destructuring#2022-05-3108:54prepor@U066U8JQJ yes, I came up to the similar principle. But in my case it's not so natural. I have this nested structure for syllabus:
(pco/resolver
 `dashboard-self-studies-completed
 {::pco/input [:syllabus.situation/id :user/id]
  ::pco/output [:dashboard.self-studies/completed]})

(pco/resolver
 `syllabus-situations
 {::pco/input [:syllabus.module/id
               (pco/? :user/id)]
  ::pco/output [{:syllabus/situations [:syllabus.situation/id
                                       :user/id]}]})

(pco/resolver
 `syllabus-level
 {::pco/input [:syllabus/level
               (pco/? :user/id)]
  ::pco/output [{:syllabus/modules [:syllabus.module/id
                                    :user/id]}]})
syllabus is completely unrelated to the user and I want to keep resolvers this way. But in some queries I want to have data specific to some part of the syllabus AND some user (like :dashboard.self-studies/completed) So I'm adding an optional input user/id to every resolver in upper levels and forward to the downstream. It's working, but it's a hack.
#2022-05-3112:20HukkaI don't think I quite understand, but if you have syllabus entities that are independent from users, and user entities, and somehow users "own" or "have" some relationship to the syllabus but with extra (meta)information added, then this relationship is it's own entity that you could model#2022-05-3112:21HukkaSo that you could query somehow like {:user [{:syllabus-relationship [:extra-information {:syllabus [:common-data-for-syllabus]}]}]}#2022-05-3112:28Bjƶrn EbbinghausThat is what I did with :room+user/name above.#2022-05-3112:29HukkaYeah, akin how many-to-many relationships would be modelled in a relational DB#2022-05-3116:23Joe R. SmithDoes Pathom-Datomic support walking attributes backwards, e.g., given a :user/accounts ref attribute, getting the user for an account using :user/_accounts ? It doesn't look like it does.#2022-05-3119:37wilkerluciogotta double check, but I think you are right, at same time I think it shouldn't be hard to add, we just need for each ref attribute, generate the reverse during the schema analysis part of the process#2022-05-3116:25nivekuilI want to get pathom-viz connected to an http parser. is there an example setup? tried giving it the fulcro /api endpoint but that doesn't seem to work#2022-05-3116:34wilkerluciowhat pathom version are you using?#2022-05-3119:40wilkerluciofor it to work with Pathom 3 you need to setup the boundary interface at the border, this tutorial contains a full working example of it: https://pathom3.wsscode.com/docs/tutorials/serverless-pathom-gcf#2022-05-3119:40wilkerlucioalso, for fully featured, it requires transit with proper setup for the Pathom 3 types encoding (also demonstrated in the tutorial I mentioned)#2022-06-0100:01nivekuilcool, I will take a look at that tutorial. getting tired of emacs freezing for 10 seconds every time an error happens because it can't render the HUGE env I have#2022-06-0100:02wilkerluciohave you tried using #portal or something like it? I find its a game changer to work with large data structures#2022-06-0100:05nivekuilI haven't, is that what you're using to read pathom errors?#2022-06-0100:05wilkerlucioportal is something that's basic on my workflow nowadays, any time I want to inspect data I use it šŸ™‚#2022-06-0100:06nivekuildo you output anything to the cider-error buffer (if you're still using emacs) or does everything go to portal#2022-06-0100:07wilkerlucioits a different channel, you send things to portal via tap>#2022-06-0100:08nivekuilah, I just want emacs to stop freezing tbh.. basically feels like I'm recompiling my program every time an error happens inside of a pathom resolver with how slow it gets#2022-06-0100:08wilkerlucioI use intellij, which Portal has a plugin to, so I can have it baked inside my editor, but it has options that you can run it as a stand alone app#2022-06-0100:08wilkerlucioso you can see my REPL history on the lower right, and portal in the top right#2022-06-0100:09wilkerlucioits an UI that allows me to interactively navigate though large data structures, has different views you can use (because the best view depends on the kind of data you are looking at)#2022-06-0100:10nivekuilso if you do p.eql/process env [:broken-attribute] does that print the whole error in the lower right repl?#2022-06-0100:10wilkerlucioyup#2022-06-0100:10wilkerlucioonly what I tap> goes to portal#2022-06-0100:10nivekuiland it doesn't lag? :O#2022-06-0100:10wilkerlucionope#2022-06-0100:10nivekuilon a multi-MB datafied env anyway#2022-06-0100:11nivekuilI think it's probably all on one line which emacs hates#2022-06-0100:11wilkerluciono, its connected to your REPL, so it can defer loading parts of data in the UI as you navigate#2022-06-0100:12nivekuilI mean the intellij repl, I can see how portal can navigate known structured data well, it's the huge stuff that pops up accidentally that kills me#2022-06-0100:12wilkerlucioI rarely have performance problems with the standard REPL on IntelliJ too, it also has a button where you can cancel if things starting to print forever#2022-06-0100:13wilkerlucioits just that even it it can print fast, still quite messy to read the data there (when its large)#2022-06-0100:13wilkerlucioone detail there is that the way Cursive does the highlight on the REPL window uses some library that can do it partially (it doesn't require knowing the end of the contents to colorize it)#2022-06-0100:14wilkerluciobut a quick tip, if you just want to ignore the output, you can run something like: (do (p.eql/process env ...) nil), so you don't get the output and prevent the issue#2022-06-0100:15wilkerluciobut if you want to look at it, then you still need to place it somewhere else (like portal) with: (tap> (p.eql/process env ...)) (tap just returns true or false)#2022-06-0100:16nivekuilthe problem is not the output, just in debugging. I want to see the error thrown in my resolver but pathom will also include the entire env in the causes, so cider tries to write all that and it explodes#2022-06-0100:17nivekuilcan you send exceptions automatically to portal? seems pretty easy for a plugin#2022-06-0100:17nivekuilyeah emacs font-lock is notoriously slow, definitely not lazy. maybe today's the day I finally try tree-sitter#2022-06-0100:17wilkerlucioyou would have to capture it somehow, easy enough to make a macro that handles the whole thing#2022-06-0100:18wilkerluciobut are you using strict or lenient mode?#2022-06-0100:19wilkerluciobecause on strict mode the errors shouldn't be that huge I guess#2022-06-0100:19wilkerluciobut I dunno how emacs handles exceptions#2022-06-0100:19wilkerluciothis is what I see in my REPL when I get a pathom exception#2022-06-0100:21nivekuilit's only on strict mode that pathom will try to print it and it gets slow I think#2022-06-0100:22nivekuilif you have a big query with big params then ::pcp/graph gets huge#2022-06-0100:22nivekuile.g. fetch the HTML of a page and store that in an attribute#2022-06-0100:22nivekuilhit 1000 sites with that in one query#2022-06-0100:22wilkerluciothats what I mean by I dunno how emacs handles exceptions, its showing automatically the ex-data of it?#2022-06-0100:23wilkerluciobecause on Cursive, it just prints the error message, so I never get anything big from it#2022-06-0100:23nivekuil(pa/q api [:eouoeeu :oeueu :eeoueoue {:>/uoeueo [:oeueou]}])#2022-06-0100:23nivekuil#2022-06-0100:24wilkerlucioyup, seems like it does automatically tries to print the ex-data indeed šŸ˜›#2022-06-0100:24nivekuilso with intellj how does it work? is the last exception saved somewhere, then you can throw it into portal?#2022-06-0100:24wilkerlucioand yeah, ex-data is big to allow detailed handling, but can be a mess if you editor tries to get it out every time#2022-06-0100:25wilkerlucioClojure itself already saves the last error as *e, so you can start from it#2022-06-0100:26wilkerlucioto be honest I rarely need to check on it, its more an exceptional case than the norm, usually the error message is enough for me to work from#2022-06-0100:26wilkerlucio(also I can click on the line to get just the stack trace, that's the thing I use more often)#2022-06-0100:28nivekuilhmm yeah there's no middle ground in cider between seeing the message of the immediate exception, vs seeing everything of the entire chain#2022-06-0100:29nivekuiland it looks like printing big exceptions is an old issue that probably isn't getting fixed https://github.com/clojure-emacs/cider/issues/2936 https://github.com/clojure-emacs/cider/issues/1873#2022-06-0100:31wilkerlucioas a workaround, you may want to have a fn wrapping the thing, so you cna for instance for the print of just the message, instead of eagerly printing the whole ex-data every time#2022-06-0100:31wilkerlucioso you can iterate faster#2022-06-0102:06nivekuilbrilliant idea! life is better now thanks šŸ™‚#2022-06-0108:38nivekuilanyone plug pathom 3's traces into opentelemetry/zipkin? I already have traces set up with mulog but not sure how to integrate those with pathom's built in tracing#2022-06-0109:49danierouxI haven't look at this yet, but I am very interested in this too.#2022-06-0109:28roklenarcicwhen creating a resolver, what should ::pco/params contain? A vector of param keywords? [:id] ?#2022-06-0111:49dehliparams is only used for mutations so you shouldn't specify that key. ::pco/input is what resolvers use and it is a vector of keywords (or maps if you wanted to specify nested inputs)#2022-06-0112:12wilkerlucioresolvers can get params too, the format is EQL, same as input and output, the description doesnt affect anything in terms of running, but can be used in the future to support tools like pathom viz#2022-06-0112:15dehliOh that's right! I've used them with things like pagination for a resolver. Forgot about that.#2022-06-0112:21nivekuilwhat's a good way to invalidate a resolver cache? the cache key computation looks a little complicated https://github.com/wilkerlucio/pathom3/blob/a1023e9fdadbf36e6cff0e4c31d29b55ce2f0d8c/src/main/com/wsscode/pathom3/connect/runner/parallel.cljc#L320#2022-06-0112:55dehli
(swap! (::pcr/resolver-cache* env)
       dissoc
       ['fully.qualified/op-name
        {:resolver/input true}
        {}])
I've used the above code to invalidate a resolver cache. The empty object as a 3rd argument is b/c the resolver I was invalidating didn't take params. If your resolver does use params, you'd need to also take that into account.
#2022-06-0112:59nivekuilwhere are you getting :resolver/input from? I assumed that would be the same input you destructure in the resolver, but the extra stuff going on in that function I linked gives me pause#2022-06-0113:02dehli:resolver/input is just an example of what the input map could be. If you had the following resolver
(ns my.namespace)

(pco/defresolver my-resolver
  [{input :my/input}]
  {:my/output (str "out: " input)})
and you wanted to invalidate the cache for input of "foo" you would do:
(swap! (::pcr/resolver-cache* env)
        dissoc
        ['my.namespace/my-resolver
         {:my/input "foo"}
         {}])
Does that make sense?
#2022-06-0113:11nivekuilit's not clear to me that
input-data      (pfsd/select-shape-filtering entity (pfsd/merge-shapes input optionals) input)         input-data      (pcr/enhance-dynamic-input r-config node input-data)
are just no-ops and you can do it that simple way, or if they need to be accounted for, which seems like exposing my code to pathom internals
#2022-06-0114:56wilkerlucio@U797MAJ8M do you want to do a precise invalidation (for one specific input/params set) or looking for to clear the cache for all entries for a specific resolver?#2022-06-0114:57wilkerluciobecause if you are looking for the later, I suggest using a separete cache store for it (https://pathom3.wsscode.com/docs/cache#custom-cache-store-per-resolver), this will make trivial to clear the whole cache on that store (just reset it to {})#2022-06-0114:59wilkerluciojust remember that to make it work as the default cache store, you will also need a plugin to reset that store for each request, otherwise it will behave more like a durable store (in the sense it will live though multiple requests, instead of per request)#2022-06-0114:59wilkerlucioyou can use this built-in plugin to help creating a new cache store for that name on each request: https://pathom3.wsscode.com/docs/built-in-plugins#extend-environment#2022-06-0115:01wilkerlucioanother thing that you may find interesting, is the config ::pco/cache-key, you can use this to provide a function that generates the cache key for that resolver, instead of using the standard [op-name input params] form#2022-06-0115:03wilkerluciothe function on ::pco/cache-key will take [env input-data] as arguments#2022-06-0115:21nivekuilah, cache-key is what I want, thanks#2022-06-0115:23wilkerluciokeep in mind that if you don't change the cache store, you are sharing it in a flat map with the cache of every other resolver, so make sure you are not doing anything that may conflict cache keys#2022-06-0115:35wilkerlucio#2022-06-0318:00sheluchinhttps://pathom3.wsscode.com/docs/indexes/#index-io > This is a good index to create auto-complete features in editors for Pathom integration. Any examples of this in the wild yet?#2022-06-0318:14wilkerluciohttps://github.com/wilkerlucio/pathom3/blob/bda89b7e437454320d1fc446a63b6585485fdc1c/src/main/com/wsscode/pathom3/connect/indexes.cljc#L369-L384#2022-06-0318:14wilkerluciothis is what Pathom Viz uses#2022-06-0318:17sheluchinThanks @U066U8JQJ. Do you know of any cases of someone integrating the index in an editor or IDE?#2022-06-0318:17wilkerlucionot yet, but I've considered in the past, would be awesome for example to have contextual auto-complete when writing the query part of Fulcro components#2022-06-0318:19wilkerlucioalso, there is this simpler version if you don't care about nested possibilities: https://github.com/wilkerlucio/pathom3/blob/bda89b7e437454320d1fc446a63b6585485fdc1c/src/main/com/wsscode/pathom3/connect/indexes.cljc#L329-L346#2022-06-0318:27sheluchinThat's good, I didn't know about that second fn. I'm thinking of adding some custom Reveal actions to make Pathom-based data exploration possible by clicking around in the Reveal window.#2022-06-0318:29sheluchinIntegration right in the editor would be nice too, my vimscript just isn't strong enough to pull that off, I don't think. Adding Pathom awareness to Reveal might be a low hanging fruit.#2022-06-0318:58wilkerlucio@UPWHQK562 an interesting option with Reveal/Portal is to use the Datafy API, Pathom Smart Maps have native support for it, so if you log a smart map, you can use the clojure Nav protocol (which is supported by Reveal) to navigate though the data#2022-06-0319:00wilkerlucioI made a video about it a long time ago: https://www.youtube.com/watch?v=n_MJOKEqqnM#2022-06-0414:31sheluchin@U066U8JQJ very cool. I'll try it out. I guess I missed it in the https://pathom3.wsscode.com/docs/smart-maps/#datafy.#2022-06-0403:05JAtkinsShort tome: Has anyone thought about using Pathom Resolvers + Mutations + Specs to define the API of your service, then letting the computer generate the UI? I've had the idea in the back of my mind for a long time that UI sucks. It's bad for us as programmers when we have to build an app across 5+ form factors (more when you consider the possibility of AR, VR, or voice assistant applications). It's bad for the consumer when we develop a subpar UI. It's bad for the consumer when we build a good UI, but it's not talored to how individuals use it. For example I use about 5% of the potential functionality of my IDE, but it's extremely tedious ranging to impossible to customize the UI for myself. Yes, keyboard bindings exist but for the most part let's ignore that. Most casual users won't learn keybindings. And, that ignores that I'd like to customize the application state I can view. OK, the problem established, here's a possible solution: stop writing UI. Instead, write api descriptors so an agent can design a UI for you, knowing the capabilities of your application. Early versions of an agent will require massive amounts of hints, but that's probably still easier than writing UI. Having an agent (ai maybe?) would also allow input from individual users to the system with hints as to how they want to view the app. "show the buttons in this order. " "show the status of this in that box." etc. Pathom + Specs might be enough information on the API side, but maybe it still comes up a bit short on the static analysis side.#2022-06-0403:49nivekuilsort of sounds like hyperfiddle?#2022-06-0404:45JAtkinsThat looks sick. It sounds like this, but with all the extra bits I though might be insane :) (e.g. UI components as resolves)#2022-06-0405:00nivekuilI think using pathom for this is more interesting though, since hyperfiddle is afaik coupled to datomic's schema#2022-06-0415:24dvingoYES YES YES I've had very similar thoughts but replacing spec with malli (https://github.com/dvingo/malli-code-gen) The main reason being that in order to programmatically and dynamically generate and parse specs you need macros. Malli schemas are easily parsed at runtime (and since starting work on that repo there is now malli.core/ast which makes this trivial ) The direction I was thinking is having a public DB/hashmap of malli ref schemas (like :com.twitter/tweet) to a fully qualified symbol representing a function that can take a hashmap that validates against the schema and produces a ReactNode type (or output for other UI libs) https://github.com/dvingo/malli-code-gen/blob/main/thoughts.md#visionary-ui-idea The pathom part is that the malli schema would generate the output vectors for pathom and then you have a handful of generic resolvers and mutations that interact with specific storage (same idea fulcro RAD uses). I've been iterating on this a little bit since, in an app but not fully there yet.#2022-06-0417:07JAtkinsSo, another interesting thought is if you have specs (malli or otherwise), it might be useful to give the agent building blocks to work with. For example, you might say the :int spec maps to a :slider. Even cooler, you can say [:int {:min 1 :max 9}] -> [:slider {:min 1 :max 9 :step 1}]. To achieve max coolness you can totally have more than 1 UI element mapped to types. e.g. {#{:int} -> #{:slider :text-input}}, or you can have that type -> ui mapping handled by pathom. The true advantage would arise when you have stuff like maps or graphs that could have many different implementations for views of the same data. Also, if you put in data source translation pathom resolvers (e.g. resolver [::graph1/location] [::graph2/location] ), the UI suddenly has way more options for what it can show.#2022-06-0418:30Thomas Moermanhttps://github.com/fulcrologic/fulcro-rad#2022-06-0418:31JAtkinsHave used it. Not quite what I want. It has too much manual code still. But similar vein I guess#2022-06-0621:27dvingoRegarding concrete implementations: This talk has a code snippet showing an idea of how you can abstract over UI implementation: https://youtu.be/BNkYYYyfF48?t=1321 This talk about reusable re-frame apps landed on a similar pattern using integrant https://youtu.be/b_uum_iYShE?t=1594#2022-06-0917:04tony.kayMuch bigger fish (i.e. Microsoft, Apple, and many many many others) have been trying to crack this very nut for decades. Not only would you like to not be writing it over and over again, companies would like to not be paying you to write it over and over. So, I think it should be sobering to you that: A. If we’re ever successful at that, a lot of us will be out of work. B. Every corporate interest on the planet would gladly pay for such a system, because it would be way cheaper than hiring us to write it. I’m not trying to discourage you, but this is not a trivial problem. ā€œLet’s just configure it via this thing (XML, json, malli), and we’ll magically have UI!ā€ It is the dream (or nightmare if you like having a job writing UI šŸ˜„ ) When I wrote RAD I just accepted at face value that this is a very hard problem, and that the results (starting with things like Visual Basic and drag-and-drop UI builders, all the way through modern-day things like XCode’s app building tools) always require a lot of customization. Your business is going to want to skin it this way or that, and there are going to be real performance problems that require trade-offs and tweaks. IMO, you need a system that can dump out something useful, but also not get in your way when you need to customize it (and you will need to customize it). Saying that RAD requires ā€œtoo much manual codeā€ is puzzling. It literally allows you to configure common UI as pure declarations, but then gives you an escape hatch when simple code generation is insufficient. From my perspective what you’re saying is ā€œthe generated code isn’t good enough, so I have to keep customizing itā€. Yep. That’s the problem. The solution from my perspective is: make it easy to do that customization. RAD has huge amounts of room for improvement (including making something that can generate complete applications from data, which it is designed to allow, but I’ve not had time to write). But I don’t have the illusion that it’ll ever not suck in some way. It’ll suck in plenty of ways, just like all the other stuff out there. You’re right to say ā€œcan’t we do more with this declared data/schema than we’re doing?ā€œ. Yes, we can. Yes, we should. The complete solution, I guarantee you, isn’t in data and API specs. You have to say, somewhere, things like what color something should be, and how many things to put on a row, and what to do when the text doesn’t fit in a box. Declaring those overrides and customizations at their location of use (e.g. the report, form, etc.) is a good solution, IMO, because you will have different opinions of what those overrides are based on their contextual use. I.e. say I have an address entity in my database. In one context (perhaps using location from browser or phone) I can fill in most of the address for them, so I only want to ask them for their street. In the UI, how do you tell it that you don’t need all the fields in that context? How do you make the validation understand that an address, in that context, needs only the street? Do anything but the most trivial thing and you’ll find that the database context very often differs from the usage (UI) context. Anyway, just my 2 cents. I do actually hope you make some inroads or improvements, but I would suggest you study prior art with more serious intention, lest you just end up re-inventing the same old crap that didn’t work last time šŸ˜„#2022-06-0917:09JAtkinsOh, lol. It's spit balling. IMO it would require "ai magic," which I have little to no skill in.#2022-06-0917:11JAtkinsThe only real "shortfall" in rad is that it's not really very user customizable. What triggered this vein of thought was seeing smalltalk systems of yore where the client could change OS code to add buttons to their apps live.#2022-06-0917:12JAtkinsBut, I dunno how to solve that. Merge conflicts alone would be a pita.#2022-06-0917:26tony.kayā€œIsn’t very user-customizableā€??? That is a surprise to hear you say that, given that the entire system has customization escape hatches everywhere. You can take over the rendering plugin, the db integration, etc. Yeah, the rendering plugin I wrote isn’t very ā€œtweakableā€, but that’s because it is a small amount of code you can copy and paste, and then plug in exactly what you want. It’s intended to demonstrate how to make a UI builder for RAD, not be the all-encompassing solution itself šŸ˜„ The biggest shortfall, IMO, is the lack of good docs. But I’m too busy at the moment.#2022-06-0917:27JAtkinsNo, it's programmer customizable. Not client customizable.#2022-06-0917:27tony.kayAH, well isn’t that just one step removed? RAD is all run-time capable.#2022-06-0917:27tony.kaymaking a user-driven interface to emit runtime UI is something I’m doing with it, just not in the open-source space.#2022-06-0917:28JAtkinsTrue that. Maybe rad with those runtime facilities is the dram. Requires some thought.#2022-06-0917:28tony.kayJust because RAD defaults to having you define artifacts in code doesn’t mean they aren’t data šŸ™‚
#2022-06-0917:28tony.kayClojure is, remember, just data#2022-06-0917:32tony.kayBut the hard problems in that space, IMO, turn out not to be the UI generation. Data is the problem: allowing the user to build arbitrary schemas and queries, which have unpredictable performance impacts, is the real quandry. You give me a predefined schema, and a GUI builder for users isn’t all that hard. But users want more: they want the data model itself to be customized.#2022-06-0917:33JAtkinsIt'd be fascinating to see what codox copilot trained on schemas and the business description would spit out#2022-06-0917:40tony.kayYes, there are all sorts of fascinating and interesting problems here. Let’s say that you have sophisticated enough users to make a good schema (Sales Force users manage to make ends meet), and you give them a reasonable way of getting a UI around it, again, a problem that has been solved many times with varying degress of ā€œsuccessā€. Now you faced with the next hard problem: Users start dumping large amounts of data into the system, and now they want a report, which requires churning through millions of records in some way. They don’t understand indexing or search engines, but they want to ask a complex question that requires one or both. I guess you’re just wanting to spit-ball around generating UI, but from my perspective the UI is always tied to the data processing, not just the schema. So, now you’ve got to give them knobs (and implementation) for saying ā€œindex thatā€, and ā€œmake that full-text searchableā€..or else the UI doesn’t ever function to meet their needs.#2022-06-0917:40tony.kayBut I guess I’m off on a rant šŸ˜„#2022-06-0917:42JAtkinsHaha šŸ™ƒ Yes, there's a reason like you said M$ hasnt announced this product#2022-06-0917:42tony.kayBut I’m on the rant because these are the things I was thinking about when designing RAD. Making an open and extensible system where you can plug in solutions to problems like these is what I see as a good solution to the real combinations of problems that real-life systems run into.#2022-06-0917:42tony.kayThe old ā€œIf you design a system that any idiot can use, then only an idiot will use itā€#2022-06-0917:44JAtkinsThat's a saying? How literally is it intended? Tis new to me.#2022-06-0917:44tony.kayIt’s a quote. I don’t remember who it is attributed to#2022-06-0917:44tony.kayI didn’t make it up, though#2022-06-0917:45JAtkinsIt sounds like a saying that would spread, so I dont doubt that šŸ˜‚#2022-06-0917:48tony.kayBut anyway, if you give me a system that can emit a UI, even a good UI, it is dangerous to me (because it leads to vendor lock-in and development drag) unless I can easily customize it in pretty much any arbitrary way. This, IMO, is why most GUI builders are not great: they get you up fast, then choke you to death. They have tons and tons of knobs to make things ā€œlook prettyā€, but no easy way to affect things that really matter. Oh, and few of them that are good are remotely usable by an end user. The end-user stuff is being done by places like SF with good success, but they have a target audience with focused constraints, and billions of dollars.#2022-06-0917:56tony.kayIt is also why RAD is a challenge for new (and even Fulcro) users: I haven’t supplied enough ā€œbatteriesā€ to make it a clear win on the initial take (my personal apps have tons of custom local add-ons to cross that line), the documentation isn’t complete, and most ppl won’t dig into the source to really understand how to escape from (or augment) what it generates. If they have experience with the other failures in this space it feels too risky to even try, because why go down a hard path that is possibly as bad or worse than the last thing like it you tried? I don’t blame them.#2022-06-0917:58JAtkinsthat happened to me :). We went for a rewrite with a relatively new-to-fulcro team. Rad didn't make the chopping block sadly.#2022-06-0918:01tony.kayNot surprising. Adopting new technologies is risky. I’ve been told by more than one SW developer ā€œWhy would I want to take a job in Clojure? That’s a career killer!ā€ Non-popular things are, well, um, not popular šŸ˜„#2022-06-0918:03tony.kayWhat’s funny is that adding RAD to a Fulcro project is one of those ā€œyou’ve already adopted the core…why not put it in and use it when you see a good spot for itā€ kinds of things…It isn’t like you have to have it do any UI for you, or any db stuff, or any resolver stuff. But, where it makes sense, it can help you with any of those. But I know a lot of people see it as a thing that you add and then have to use for everything…another PR problem that I have no energy to solve.#2022-06-0918:07JAtkinsIt's an empty promise till I have time (hah), but adding docs and features to RAD is next on my OSS help list.#2022-06-0918:10tony.kayalways happy for the help. Let me know when you have time and I can probably help direct your efforts more productively#2022-06-1623:20wilkerlucio@U5P29DSUS are you familiar with https://www.appgyver.com/ ? this is the closest thing I found around the kind of no-code dev environment#2022-06-1623:20wilkerlucioone of the things that make it complicated its because its not a single problem, but a bunch of them that we are trying to make a silver bullet for#2022-06-1623:22wilkerluciodesign is a big issue on itself, where things go, how things respond to size changes... I think XCode does a good job on this area#2022-06-1623:22wilkerluciobut them there is the "make it alive", integrations, loading, performing operations... those have to have integrations with the UI, and have an implementation for the actual behaviors#2022-06-1623:24wilkerlucioI believe pathom+fulcro is a good combo to design the primitives of such system, by making EQL the de facto layer between the UI and the underlying systems (that may be a server, but could also just be local stuff, the point is that it shouldn't matter for the UI layer)#2022-06-1623:26wilkerluciothem, to make it "non developer friendly" we need a whole ecosystem of things on top of it, some people can make packages of UI's (bootstrap, bulma, blueprintjs...), other people make packages of server integrations (pathom resolvers and mutations) and finally someone else can use the codeless IDE to plug all together#2022-06-0514:12sheluchinIs there a recommended approach for using a DB as a persistent cache store? Some of my attributes require a heavy DB lookup, and I'd like to save the final result to a DB key to optimize.#2022-06-0603:47lgesslerbasically you mean denormalization, right? I've had to do some, afaik there aren't any xtdb specific facilities for it, though the transaction listener API might be useful depending on your situation#2022-06-0618:38sheluchinYes, basically saving the denormalized final result to some place in the DB so it can be retrieved without traversing the graph. At the moment I'm achieving it by creating two resolvers, x and cache->x, where the latter has a higher priority and just checks the DB under for a valid cache-key record with the resolver attribute x (or whatever the name is), returning nil if not found. I could accomplish the same with a single resolver and a conditional in it, but I'm trying it this way :man-shrugging: I just wonder if there's an overall cleaner/more common approach to doing it.#2022-06-0715:33lgesslerdoh I'm sorry I thought this was in #xtdb :man-facepalming: your two ideas seem sensible to me#2022-06-0715:34lgesslermy initial reaction's that the single resolver approach makes more sense, and here's why: it's fine for pathom resolvers to be side effecting wrt database state, and you can think of cache state as just another kind of db state#2022-06-0715:47sheluchin> it's fine for pathom resolvers to be side effecting wrt database state Really? I thought I read the opposite somewhere recently - that they should be pure. Another idea I had for implementing this was to create a plugin that uses https://pathom3.wsscode.com/docs/plugins/#pcrwrap-resolve to perform they cache-key lookup and skip the resolver call if something is found. Tried it briefly, but went with the previously mentioned idea because the mechanics there are more familiar to me right now.#2022-06-0716:37wilkerlucio@UPWHQK562 there are ways to setup custom cache stores, and set that to be used by specific resolvers: https://pathom3.wsscode.com/docs/cache#custom-cache-store-per-resolver#2022-06-0717:01lgessler> Really? I thought I read the opposite somewhere recently - that they should be pure. I'm not a Pathom expert and I agree it's a good general rule to keep resolvers pure, but the question is, imo, just whether it'd be bad for a side-effect in a resolver to occur many times and unpredictably. For a properly implemented caching side effect, maybe it's OK--for a very expensive side effect or one with destructive consequences, it's probably not ok#2022-06-0717:05wilkerlucioI think its ok to side effect, as long as you understand and is ok with the implications, after all caching is a side effect that happens for resolvers all the time#2022-06-0717:08wilkerlucioanother common side effect is to log resolver duration for observability#2022-06-0717:08sheluchin@U066U8JQJ the way described in that link requires the cache to be implemented with core.cache, right? Or just using com.wsscode.pathom3.cache/CacheStore? If it's the latter, it doesn't have to be implemented using Atom or Volatile?#2022-06-0717:19wilkerluciono core cache required, thats just an example or implementing a custom CacheStore#2022-06-0717:19wilkerlucioyou just need to make your custom CacheStore#2022-06-0717:19wilkerlucioPathom extends Atoms and Volatiles to implement CacheStore, so you can use them directly as cache stores#2022-06-0717:20sheluchin> This protocol is implemented by Pathom to Atom and Volatile, so you can use any of those as a cache-store. I think this part is what I'm confused by. I don't want to use either of those for my cache store because those are in-memory, and I want to use my database as the cache store. Am I misunderstanding something here?#2022-06-0717:23sheluchinAtom and Volatile are two of the readily available options, but the protocol can be extended to use anything else as well. That's the correct interpretation?#2022-06-0717:24wilkerlucioyes, thats correct#2022-06-0717:24wilkerlucioa cache store is anything that implements CacheStore protocol#2022-06-0717:25sheluchinheh I think I made that more difficult than it needed to be šŸ˜› Thanks very much, and you as well, @U49U72C4V. I'll try that shortly.#2022-06-0717:26sheluchinMakes sense about cautious side-effects like logging and caching.#2022-06-0715:27roklenarcicHow do you handle attributes that kinda require a context to be resolved? For instance let’s say you’ve got: [:folder/id :folder/name :folder/path {:folder/files [:file/id :file/name :file/full-path]}] Here the resolver for full-path would want to have folder/path and file/name but at that point only file/name is available. Now obviously you could generate full-path when resolving folder/files, but let’s say that you want to have a separate resolver for it. That takes folder/path and file/name.#2022-06-0716:21sheluchinYou could have a full-path-resolver that specifies its input to be a nested value with all the required data, like ::pco/input [:folder/path {:folder/files [:file/name]}], and pull the necessary data out of the input to form your :file/full-path output.#2022-06-0716:29wilkerlucioas a general guide, you can never look up, only down, so in this specific case I think would be better to use the full path as the primary thing, since its trivial to compute the base name from it#2022-06-0716:29wilkerluciothat or, each level going down needs to forward some information to retain the context#2022-06-0812:38roklenarcic1. Does specifying pco/? in output vector do anything? 2. If I return ::missing for some/id that gets pushed into the next resolver as a value, how would I short-circuit that? Use pco/transform to wrap all handlers in some code that handles this? #2022-06-0818:00wilkerlucio1. no, because optionality is always on the demand side (input) never on the output#2022-06-0818:00wilkerlucio2. not sure if I understand, if you wanna say you dont have the value, dont output the key#2022-06-0818:58roklenarcicright but if I dont output the key I get an error. This is kinda a running problem I’ve got with optional stuff. Sometimes when outputting a list of items that may have 1 optional attribute I want to filter based on whether the optional attribute was requested or not (and so avoid the error by returning just the items with the optional attribute present).#2022-06-0819:00wilkerluciocan you send a repro that demonstrates the unexpected behavior?#2022-06-0819:14roklenarcicnot as much unexpected behaviour as missing feature#2022-06-0813:03roklenarcicThe weird part of the optional value story is what happens when you specify some derived attributes as optional. I have a resolver user/id => [(pco/? :github/id)] and a resolver github/id => [github/email github/username] Now the query `
[{[:user/id 11] [(pco/? :github/id)]}]
works like you’d expect for a user without github, {[:user/id 11] {}} Then I try to add another data point:
[{[:user/id 11] [(pco/? :github/id) (pco/? :github/email)]}]
And I’ll get All paths from an OR node failed. . So optional values are only really useful for ā€œleafā€ attributes?
#2022-06-0818:01wilkerlucioare you on the latest pathom3? the last case there was something I remember fixing recently#2022-06-0818:02wilkerlucioif you are, please send a repro case and I can take a closer look#2022-06-0818:58roklenarcicupdated to the latest, still same problem#2022-06-0819:14roklenarcic
(pco/defresolver res1 [_]
  {::pco/output [{:user/all [:user/id]}]}
  {:user/all (map #(hash-map :user/id %) (range 10))})

(pco/defresolver res2 [{:user/keys [id]}]
  {::pco/output [:github/id]}
  (if (even? id) {:github/id id}))

(pco/defresolver res3 [{:github/keys [id]}]
  {::pco/output [:github/username]}
  {:github/username (str "user " id)})

(def env
  (-> {::p.error/lenient-mode? true}
      (pcp/with-plan-cache)
      (pci/register [res1 res2 res3])))

(defn ask [tx]
  (p.eql/process env tx))

(comment
  ;; works
  (ask [{:user/all [:user/id (pco/? :github/id)]}])
  ;; doesn't work, error
  (ask [{:user/all [:user/id (pco/? :github/username)]}])
  )
#2022-06-0819:17roklenarcichowever it works if I do the following:
(pco/defresolver res1 [_]
  {::pco/output [{:user/all [:user/id]}]}
  {:user/all (map #(if (even? %) {:user/id % :github/id %} {:user/id %}) (range 10))})

(pco/defresolver res3 [{:github/keys [id]}]
  {::pco/output [:github/username]}
  {:github/username (str "user " id)})

(def env
  (-> {::p.error/lenient-mode? true}
      (pcp/with-plan-cache)
      (pci/register [res1 res3])))

(defn ask [tx]
  (p.eql/process env tx))

(comment
  ;; works
  (ask [{:user/all [:user/id (pco/? :github/id)]}])
  ;; works
  (ask [{:user/all [:user/id (pco/? :github/username)]}])
  )
#2022-06-0819:18roklenarcicso if I reduce resolver path by 1level it works#2022-06-0915:02wilkerluciothanks for the demos, I'll quite busy today but I'll have a closer look at it tomorrow#2022-06-2201:42wilkerluciohello @U66G3SGP5, sorry the long delay, I have tried now the examples, but they seem to behave correctly to me#2022-06-2201:42wilkerlucioif you use strict mode, all the asks work fine (no exception thrown)#2022-06-2201:42wilkerlucioon lenient mode, you get the error detail because you will always get all errors there, but in terms of result they are equivalent#2022-06-2201:43wilkerluciomakes sense?#2022-06-0903:53JAtkinsis there a way to tell pathom to pull all computable options for a key? e.g.
(defresolver spelling-error
  [{:keys [:text]}]
  {:errors [...]})

(defresolver not-actually-a-string-error
  [{:keys [:text]}]
  {:errors [...]})
I'd like pathom to compute both and concat the results. Clearly, this doesn't work, but is there a way to get this capability?
#2022-06-0908:16roklenarcicWhy are they a separate resolver? I don;t think you can do that and I don’t think it will work like that in the future#2022-06-0909:02nivekuilyou could make a super-resolver that directly calls those resolvers and concats them#2022-06-0909:39JAtkins@U66G3SGP5 Yeah, doesn't work like that now either šŸ˜›. Different resolvers because this is part of a plugin concept - allowing more outputs with no prior knowledge from the owner or the query-er. I could implement it with a super-resolver that has a registration mechanism for all providers, but that requires knowledge that a key might have many providers... Not terrible but still requires work. on the query-er side I don't want to know before hand that I'm looking for :spelling-errors and :wrong-type-errors .#2022-06-0917:16JAtkinsThe solution might be to write a custom pco/env or whatever it's called that takes resolvers and makes a mutation. If it sees many impls of the same key, it could write the super resolver into the env#2022-06-1622:52wilkerluciothis goes against a core premise of pathom that is: each attribute only gets resolved once#2022-06-1622:54wilkerluciocould be possible, but I'm afraid it would add a lot of extra complexity to the internals#2022-06-1708:35JAtkinsYeah, I’ve just written a custom layer on pathom-indexes/register to find duplicates, prune them, and make a master resolver out of it#2022-06-0915:01roklenarcicIf I’ve got more than one parameter set in the query, how do I get the other one? e.g.:
(pco/defresolver res1 [env _]
  {::pco/output [:user/first-name :user/last-name]}
  (println (pco/params env))
  {:user/first-name "Me" :user/last-name "OK"})

(query ['(:user/first-name {:x 1}) :user/last-name])
will print {:x 1} as expected but this will print the same:
(query ['(:user/first-name {:x 1}) '(:user/last-name {:y 2})])
#2022-06-0915:03wilkerlucioparameters are per resolver call, most of the time they are used to filter a collection so there is only one attr there#2022-06-0915:04wilkerluciofor now the solution is to break those in separate resolvers #2022-06-0915:05roklenarcicparameter could be something trivial like :uppercase? and that could go onto multiple attributes#2022-06-0915:05roklenarcicbut thanks for clarifying#2022-06-0917:22wilkerlucioI would not recommend, feels like pushing some aesthetic things to the data process, that to me sounds something you wanna do after the fact, but if its a business thing, I think would be better to have a derived attribute instead of making it a param#2022-06-1116:34nivekuildid some more digging on huge pathom error messages blowing up emacs. I think the problem may be that ex-message is huge here:
(pco/defresolver bad []
  {:a (/ 1 0)})
(def env (pci/register bad))
(try @(p.a.eql/process env  [:a])
     (catch Exception e
       (throw (ex-info (ex-message e) {}))))
#2022-06-1116:34nivekuilyou don't expect ex-message to ever be that big right?#2022-06-1116:38nivekuilhttps://github.com/wilkerlucio/pathom3/issues/142 can't post the error on slack, too big#2022-06-1610:10JAtkinsHi all, Currently I’m working with entity missing exceptions for our domain model. Theoretically, it’s pretty simple to add :entity/accessible? keys to each of our xxx-by-id resolvers, but it’d be nice to instead extend the ::pathom-runner/attribute-error map. The intended data shape was
{:xxx/id ... ;; Entity map

 ::pathom-runner/attribute-error {:xxx/id :unreachable}}
However, no plugin in pathom allows extending attribute-error in this way, except for the root run node plugins as far as I can tell. Is this data shape possible with plugins? Neither ::pathom-runner/wrap-resolver-error nor ::pathom-error/wrap-attribute-error are capable of making this shape.
#2022-06-1620:46wilkerlucioso you wanna change the format of the error, let me take a look to see if I can stop some other extension point, otherwise we may add one more#2022-06-1620:50wilkerlucioI see the only place that attaches those errors in is this fn: https://github.com/wilkerlucio/pathom3/blob/db656982f186cedc434786be05c0c28229b85056/src/main/com/wsscode/pathom3/error.cljc#L96#2022-06-1620:51wilkerluciojust by looking at the code seems like ::wrap-attribute-error should be able to update that, no? I can give a try here to double check#2022-06-1620:55wilkerlucioI have an example here, please let me know if its sufficient:#2022-06-1620:55wilkerlucio
(ns com.wsscode.pathom3.demos.attribute-error-wrap
  (:require [com.wsscode.pathom3.connect.operation :as pco]
            [com.wsscode.pathom3.connect.indexes :as pci]
            [com.wsscode.pathom3.interface.eql :as p.eql]
            [com.wsscode.pathom3.plugin :as p.plugin]))

(pco/defresolver error [{::keys []}]
  {:foo (throw (ex-info "lala" {}))})

(def env
  (-> {:com.wsscode.pathom3.error/lenient-mode? true}
      (pci/register error)
      (p.plugin/register
        {::p.plugin/id 'error-thing
         :com.wsscode.pathom3.error/wrap-attribute-error
         (fn [attr-error]
           (fn [entity k]
             (let [err (attr-error entity k)]
               (:com.wsscode.pathom3.error/cause err))))})))

(comment
  (p.eql/process env [:foo]))
#2022-06-1620:55wilkerlucioresult:
=>
{:com.wsscode.pathom3.connect.runner/attribute-errors {:foo :com.wsscode.pathom3.error/node-errors}}
#2022-06-1622:11wilkerluciotaking a second read I see I may have misinterpreted the issue. is correct that what you want is to affect the error on the parent level according to some definition inside :xxx/id?#2022-06-1708:29JAtkinsSo, it’s a bit more complicated unfortunately. I’d like to use an ident join to perform root loads in fulcro, but if the entity is missing return in the error key :unreachable
{[:xxx/id ...]  
  {:xxx/id ... ;; Entity map

   ::pathom-runner/attribute-error {:xxx/id :unreachable}}}
But, since :xxx/id is not being resolved by the planner, it’s not an option for the wrap-attribute-error plugin.
#2022-06-1620:44roklenarciclooking at pathomviz useage docs I think there’s a step missing.. I think you need to expose parser via an endpoint that uses Transit, am I right?#2022-06-1620:46wilkerluciodepends, if you wanna use the HTTP connection yes, but otherwise you should use the pathom viz connector: https://github.com/wilkerlucio/pathom-viz-connector/#2022-06-1620:46wilkerlucioits recommended to use the connector during dev, the http is more targeted for staging/prod environments#2022-06-1708:00roklenarcicit runs now, but I’d love to fix #object[Object [TaggedValue: unknown, #clj-time/date-time ā€œ2020-02-13T10:20:14.000Zā€]],#2022-06-1714:17wilkerluciothis is the same issue with Fulcro Inspect, because they run their own environment it would have to know about specific libraries like clj-time to handle that#2022-06-1714:17wilkerlucioso it uses the generic TaggedValue for custom types#2022-06-1620:47wilkerlucioand to everyone here, sorry I'm a bit slow these weeks, I'm on vacation and trying to stay away from computer a bit, I'll be fully back by June 27 šŸ™‚#2022-06-2116:32sheluchinI have a request that fails only when A and B are requested together, but works when either A or B are requested. Can anyone suggest how to troubleshoot the problem?#2022-06-2116:36Bjƶrn EbbinghausWhat resolvers do you have? How does the Query look like? Have you checked if the output of the resolvers is properly defined?#2022-06-2118:11sheluchinHmm, that all seems okay to me, but I do notice that when I include both A and B, the RAD DB adapter resolvers are not being run, but when I eliminate either A or B, they do run :thinking_face:#2022-06-2214:40sheluchinI still haven't really figured out why it's happening, but I did figure out that if I request C along with A and B, it all works lol. They should all be accessible simply from :A/id, so all the more confusing.#2022-06-2309:06Bjƶrn EbbinghausHm. Sorry, but I don't have a lot of experience as of how RAD generates resolvers.#2022-06-2309:07Bjƶrn EbbinghausIs this with pathom2 or 3?#2022-06-2315:04sheluchinIt's Pathom3. Indeed, I think some of the issue is in the resolver generation. I had some strange issues, including https://github.com/wilkerlucio/pathom3/blob/d5b86d8e6d7a159ef0ef970de02c4145a82e030b/src/main/com/wsscode/pathom3/connect/runner.cljc#L358= that I was only able to fix by altering the https://github.com/roterski/fulcro-rad-xtdb/blob/07d713b64104ad588ab85f7dc4ac873c4927bba0/src/main/roterski/fulcro/rad/database_adapters/xtdb/generate_resolvers.clj#L121 to stop batching. At the moment I'm not going to pursue a fix beyond just including C in my query to make it all work, even though I don't need C. I would like to understand what can cause such a problem; it seems illogical to me. I need to get better at interpreting the planner graphs.#2022-07-0116:19sheluchinI'm still having some issues with this, as described in the thread root. I'm looking at the plan and it looks like what is happening is that at some point A and B get merged together, which causes a failure when trying to get both of them. I attached the graph; I need to get both 1 and 3, but after Merging sibling resolver calls to resolver, I think one of them gets cancelled out. Any tips for dealing with this?#2022-07-0511:28wilkerluciohello @UPWHQK562, can you make a repro for this? I haven't been using RAD so I'm not sure whats going on#2022-07-0516:42sheluchin@U066U8JQJ I took some time to try to make a minimal repro for it the other day but couldn't reproduce outside my environment šŸ˜• I guess it's my data or something else that is responsible. Can you please suggest which area of the docs/source I should study to improve my ability to troubleshoot Pathom issues?#2022-07-0620:37wilkerlucionot really sure at this point, I always try to reduce the query as much as possible, so we can get to the root cause, but so far with the info here its still vague to me#2022-07-0116:19sheluchinI'm still having some issues with this, as described in the thread root. I'm looking at the plan and it looks like what is happening is that at some point A and B get merged together, which causes a failure when trying to get both of them. I attached the graph; I need to get both 1 and 3, but after Merging sibling resolver calls to resolver, I think one of them gets cancelled out. Any tips for dealing with this?#2022-06-2314:10mbjarlandhmm...so I'm trying to get pathom-viz to work on Ubuntu 22.04 with a nvidia gtx 1060 card and receive the following:
ā”€āž¤ pathom-viz 
APPIMAGE env is not defined, current application is not an AppImage
INFO [com.wsscode.node-ws-server:136] - Websocket Server Listening on port 8240
INFO [com.wsscode.node-ws-server:75] - Starting express
[999214:0623/160854.477193:FATAL:(415)] GPU process isn't usable. Goodbye.
[1]    999214 trace trap (core dumped)  pathom-viz

ā”€āž¤ 
#2022-06-2314:12wilkerlucioI'm not familiar with the Ubuntu setup, but looking for generic Electron issues (which Pathom Viz uses) I found this: https://github.com/ebkr/r2modmanPlus/issues/735#2022-06-2314:12mbjarlandafter some digging it seems that the vaapi which pathom-viz is using (?) is not entirely supported on nvidia cards...has anybody run pathom-wiz on an nvidia card?#2022-06-2314:12mbjarlandaha#2022-06-2314:14mbjarlandyou the man! : ) Thank you, the --no-sandbox switch seems to fix the issue and get the UI up and running#2022-06-2314:14wilkerluciocool, glad to hear it works šŸ™‚#2022-06-2314:15mbjarlandand thank you for a new way of dealing with data requirements on the client, just watching your "Data Navigation with Pathom 3" video and it's a bit of an eye opener#2022-06-2314:16mbjarlandwriting a pathom server for the star wars api as an exercise, should be fun#2022-06-2314:18wilkerluciohope so, let me know if you have any hiccups in the process#2022-06-2314:18mbjarlandso far so good : ) will do#2022-06-2314:35mbjarlandum...since you seem to be online...I know I saw this somewhere and apologies for a newbie question, but given that I have say two resolvers person-resolver and film-resolver, how do I tell pathom that the items in a collection :person/films [1 2] are actually the entities returned by (film-resolver {:film/id 1})?#2022-06-2314:35wilkerluciono dummy questions šŸ™‚#2022-06-2314:36wilkerlucioin this case, the trick is to make each entity a map instead of just the number, so instead of :person/films [1 2] you use :person/films [{:film/id 1} {:film/id 2}]#2022-06-2314:36wilkerluciobecause the data itself is the context for further processing#2022-06-2314:38mbjarlandaha, ok will write some helpers to make that happen#2022-06-2314:42mbjarlandand that makes total sense#2022-06-2617:12mbjarlandGeneric noob question about pathom. So I’m playing with mapping the star wars api with pathom and also connecting imdb data via the films represented in the swapi. My question is general, but to use the swapi as an example: the swapi has a ā€œfilmā€ entity with a bunch of attributes, I would like to connect the swapi film entity with the imdb film entity and pull in some film information from imdb (like user rating etc) to ā€œdecorateā€ the swapi film entity. How would you typically model this in pathom? I’m guessing you would fully qualify everything and do swapi.film/id and imdb.film/id etc and then connect the two. This essentially makes the two film entities totally separate things, i.e. not so much decorating as just querying a different entity. I guess you would then add something like a synthetic entity swapi.film/imdb.film…but I’m not sure this feels right. So how do you typically model two different sources providing information about essentially the same thing (let’s say the 1977 star wars movie)?#2022-06-2617:17nivekuilI would just do them as separate entities, same way https://www.wikidata.org/wiki/Q17738 does it. How you want to combine them into a "canonical" entity is up to the client really#2022-06-2617:25wilkerluciogreat question! the first step is to map them separate, like you just said, the second one would be to connect those entities, one way to do it is via ID's, would be great if the SWAPI already had that info, but since it doesn't we can add this mapping ourselves (a bit manual, but considering its less than a dozen items, works fine), with something like:
(def swapi->imdb-id-mapping
  {1 "tt0076759"
   2 "tt0080684"
   3 "tt0086190"
   ; ...
   })

(pco/defresolver swapi->imdb [{:keys [swapi.film/id]}]
  {:imdb.movie/id
   (get swapi->imdb-id-mapping id ::pco/unknown-value)})

; inverse direction

(def imdb->swapi-id-mapping
  (set/map-invert swapi->imdb-id-mapping))

(pco/defresolver imdb->swapi [{:keys [imdb.movie/id]}]
  {:swapi.film/id
   (get imdb->swapi-id-mapping id ::pco/unknown-value)})
#2022-06-2617:26wilkerluciothis will allow you to query both entities at the same level, like:
(p.eql/process env 
  ; via swapi ID
  {:swapi.film/id 1}
  [:swapi.film/title :imdb.movie/rating])

(p.eql/process env 
  ; same query, via imdb id
  {:imdb.movie/id "tt0076759"}
  [:swapi.film/title :imdb.movie/rating])
#2022-06-2617:37wilkerlucioa third step you might want to take is to abstract those specific services name away from your client, for example, if we were creating the "Star Wars Fan Site" (`swfs` for short) service, we might want to use names like :swfs.movie/title :swfs.movie/rating, so we shield our client from having to know about the different services, this is more a design decision, in some cases you might prefer using the names strait from the services, but its a consideration to make, if a layer on top will be helpful for the case, and if so, you can use an https://pathom3.wsscode.com/docs/built-in-resolvers#aliasing, like:
(pbir/alias-resolver :swapi.film/title :swfs.movie/title)
(pbir/alias-resolver :imdb.film/rating :swfs.movie/rating)
#2022-06-2617:41mbjarlandPerfect. This is way cleaner and answers my question perfectly. I did not realize that adding two resolvers for the IDs would allow me to query the items on the same level directly#2022-06-2617:41mbjarlandThank you @U066U8JQJ!#2022-06-2617:42wilkerluciono worries, this is where you get the real leverage from the attribute modelling, because it makes trivial to "merge" entities like these, combining different sources at the same level#2022-06-2617:54mbjarlandOne more question…in general, would you advocate adding an id field to the information sent to the client or would you rather keep that inforamtion internal and just feed the client the data#2022-06-2620:15wilkerlucioif its a public id, I think its fine to keep, also you need it to make the relations#2022-06-2620:34roklenarcicIs there a way to short-circuit resolving a subtree with a plugin or something? I’ve made an attempt to that effect and it always resolves the whole tree.#2022-06-2622:28wilkerluciocan you give an example demo? what kind of flow you are looking for?#2022-06-2709:06roklenarcicbasically let’s say you’ve got a resolver that given a ::github/repo-name will return ::github/repo-id , you can see how for some names it might not have a valid return as there is not repository of that name. Then I hafve a resolver that given ::github/repo-id can produce ::github/star-count . Now obviously I don’t want to code all my resolvers that take in repo-id to check for nil input or ::missing input values.#2022-06-2710:04roklenarcicSo far I’ve tried to code around this by introducing a filter that does: 1. if input parameter contains any ::missing values skip calling the resolver and return a response constructed from pcp/expect 2. if resolver return is ::missing return a response constructed from pcp/expect 3. the response from expect is constructed by replacing all {} in pcp/expect with ::missing This kinda works, you get the shape requested by all the properties and if somewhere in the chain we stopped gettign data, the properties are show as ::missing . The main problem there is that the expects shape and EQL in general don’t specify whether the return is a collection or a single item, the resolver can return either. So when I do something like this:
{:projects/current-user [:github/id :github/name {:github/contributors [:contributor/name]}]}
Those projects that don’t have a linked github repository will get the wrong shape:
{:projects/current-user
       [{:github/id 1
         :github/name "test 1"
         :github/contributors [{:contributor/name "guy 1"} {:contributor/name "guy 2"}]}
        ; this project has no github link so it returns ::missing value and this result
        ; is contructed from expect
        {:github/id ::missing 
         :github/name ::missing
         :github/contributors {:contributor/name ::missing}}]}
#2022-06-2710:06roklenarcicnote that second contributors is not a vector#2022-06-2710:07roklenarcicI’ve also tried using pco/final-value to short circuit processing but that seems to do nothing#2022-06-2721:56wilkerlucioI get a bit concern about the ::missing thing, because Pathom already has semantics for it (which are: dont provide the key), adding ::missing makes you whole system in need to handle that everywhere, mixing with actual values, while by not providing a value, pathom will stop the execution (and give an error)#2022-06-2721:58wilkerlucioand to go around items that misses things, you can either make some parts of the query optional, or use lenient mode (which will allow for problems in the middle of the query, and report errors by attribute, while keeping what was resolved)#2022-07-0111:23roklenarcicThis part doesn’t work well for me. Not providing a value is not a very good solution. You’ve mentioned two options, making part of query optional and using lenient mode. I don’t find these sufficient. 1. using pco/? pco/? doesn’t work when the attribute it wraps is not the optional attribute but it is sourced by an optional attribute. Your planner is just not sufficiently capable to make this realistic. Consider this example:
(pco/defresolver a1 [{}]
  {::repos
   [{::repo "repo1"}
    {::repo "repo2"}]})

(pco/defresolver a2 [{::keys [repo]}]
  {::pco/output [::project-id]}
  (if (= repo "repo1") {::project-id 1} {}))

(pco/defresolver a3 [{::keys [project-id]}]
  {::project-name
   (str "project" project-id)})
So some repos have a project and some don’t. I did it as you suggested, omiting attribute when it is not available. Then if I use your optional query advice:
(ask [{::repos [(pco/? ::project-id)]}])
=> #:resolvers{:repos [#:resolvers{:project-id 1} {}]}
Seems it works. But it doesn’t really, when I request a different attribute as optional the planner will request all attributes leading up to it as mandatory:
(ask [{::repos [(pco/? ::project-name)]}])
=>
#:resolvers{:repos [#:resolvers{:project-name "project1"}
                                        #:com.wsscode.pathom3.connect.runner{:attribute-errors #:resolvers{:project-name #:com.wsscode.pathom3.error{:cause :com.wsscode.pathom3.error/node-errors,
.....
:error-data {:required #:resolvers{:project-id {}},
It firmly requires ::project-id even though it is needed only to compute an optional attribute. Another thing: does it work with joins? (pco/? {::some-attr [::more-attr ::more-attr]}) I haven’t tried but I suspect it doesn’t. So this pco/? is not very useful, it only really works for leaves of the query tree, and only those that don’t have optional / potentially missing inputs. I also suspect that pco/? doesn’t play well with fulcro defsc 2. Lenient mode and ignoring the errors it creates This kinda works but it is very messy: • Suddenly my missing data errors that are expected are lumped in with other serious errors. • It generates a shitload of output in REPL and it makes results hard to read • if there’s an OR node involved I get bunch of stack traces in my output and it makes for a large output, also generating that much garbage cannot be cheap. This is especially bad with large collections. I just get pages and pages and pages of errors. I have to wonder how much does that cost to create.
#2022-07-0500:19wilkerluciohi @U66G3SGP5, I tried to reproduce the issue you said with 1, but in the tests I did an indirect dependency doesn't blow up when a previous part fails, based on this example:
(pco/defresolver a [{::keys []}]
  {::a ::pco/unknown-value})

(pco/defresolver b [{::keys [a]}]
  {::b (str a " x")})

(def env
  (pci/register [a b]))

(comment
  (p.eql/process env [(pco/? ::b)]))
#2022-07-0500:19wilkerlucioyou can see, ::b will fail due to no value in A, but in this case, no error is thrown, but if you found a case where this isn't working as expected, please let me know with a repro#2022-07-0500:20wilkerlucioabout 2, I preferred to give more information at start, so you can see what you have available, but you can make a plugin to process the error and tune it down for something that makes more sense to your scenario#2022-07-0506:59roklenarcicDoes my example not reproduce the error? This is all in lenient mode btw.#2022-07-0510:06roklenarcicmy example doesn’t use unknown value#2022-07-0510:07roklenarcicrather it doesn’t include the attribute as you’ve suggested#2022-07-0510:28wilkerluciounknown value is same as not having the key in the output#2022-07-0510:29roklenarcicright…you’ve suggested that when I don’t have a value to omit the key in the return and use pco/? when doing a query#2022-07-0510:29roklenarcicI gave you an example of resolvers a1 a2 a3 where this approach produces errors#2022-07-0510:30roklenarcicYou’ve said tha tyou cannot reproduce, so I am wondering if you’ve tried this example… the one you’ve listed uses a special pco/unknown value instead#2022-07-0510:47roklenarcicI guess I have poorly communicated the issue#2022-07-0510:49wilkerluciolet me try that#2022-07-0510:50wilkerluciobecause the premisse of optional is that its ok to fail in a dependency, that should work, so I made a simples example to try it#2022-07-0510:51roklenarcicbut your example doesn’t fail to provide a dependency, does it? It provides ::a just fine it’s just ::pco/unknown-value. Does pco/unknown-value have a special meaning?#2022-07-0510:54wilkerlucioyeah, ::pco/unknown-value is same as not having the key in the output (different than nil)#2022-07-0510:54wilkerlucioI can rewrite my same example as:#2022-07-0510:54wilkerlucio
(pco/defresolver a [{::keys []}]
  {::pco/output [::a]}
  {})

(pco/defresolver b [{::keys [a]}]
  {::b (str a " x")})

(def env
  (pci/register [a b]))

(comment
  (p.eql/process env [(pco/? ::b)]))
#2022-07-0510:54roklenarcicright, now try to put another resolver in between#2022-07-0510:55wilkerlucioI just tried your demo, it doesn't trigger an error#2022-07-0510:56wilkerlucio
(do
  (pco/defresolver a1 [{}]
    {::repos
     [{::repo "repo1"}
      {::repo "repo2"}]})

  (pco/defresolver a2 [{::keys [repo]}]
    {::pco/output [::project-id]}
    (if (= repo "repo1") {::project-id 1} {}))

  (pco/defresolver a3 [{::keys [project-id]}]
    {::project-name
     (str "project" project-id)})

  (def env
    (pci/register [a1 a2 a3]))

  (p.eql/process env [{::repos [(pco/? ::project-name)]}]))
=>
{:com.wsscode.pathom3.demos.repro-opt-follow-up/repos [{:com.wsscode.pathom3.demos.repro-opt-follow-up/project-name "project1"}
                                                       {}]}
#2022-07-0510:59roklenarcicenable lenient mode#2022-07-0511:00wilkerlucioright, that's something to check, it shouldn't, but I see it#2022-07-0511:01roklenarcicyour example with just lenient mode turned on returns error#2022-07-0511:01roklenarcic
(pco/defresolver a [{::keys []}]
  {::pco/output [::a]}
  {})

(pco/defresolver b [{::keys [a]}]
  {::b (str a " x")})

(def env
  (-> {::p.error/lenient-mode? true}
      (pci/register [a b])))

(comment
  (p.eql/process env [(pco/? ::b)]))
#2022-07-0511:01wilkerlucioI though you were comparing between using strict mode with optionals vs lenient mode#2022-07-0511:01wilkerluciobecause usually on lenient mode, optionals can kind be ignored (since you still get all the payload), but I'll look into this, this cases shouldn't add errors (if I remember correctly)#2022-07-0511:09wilkerlucioand to illustrate what I was saying in point 2, this is how you can simplify the error, in this demo I'm just outputting the cause of it:#2022-07-0511:09wilkerlucio
(do
  (pco/defresolver a1 [{}]
    {::repos
     [{::repo "repo1"}
      {::repo "repo2"}]})

  (pco/defresolver a2 [{::keys [repo]}]
    {::pco/output [::project-id]}
    (if (= repo "repo1") {::project-id 1} {}))

  (pco/defresolver a3 [{::keys [project-id]}]
    {::project-name
     (str "project" project-id)})

  (def env
    (-> {:com.wsscode.pathom3.error/lenient-mode? true}
        (p.plugin/register
          {::p.plugin/id 'err-simplifyer
           :com.wsscode.pathom3.error/wrap-attribute-error
           (fn [attr-error]
             (fn [entity k]
               (let [err (attr-error entity k)]
                 (:com.wsscode.pathom3.error/cause err))))})
        (pci/register [a1 a2 a3])))

  (p.eql/process env [{::repos [(pco/? ::project-name)]}]))
=>
{:com.wsscode.pathom3.demos.repro-opt-follow-up/repos [{:com.wsscode.pathom3.demos.repro-opt-follow-up/project-name "project1"}
                                                       {:com.wsscode.pathom3.connect.runner/attribute-errors {:com.wsscode.pathom3.demos.repro-opt-follow-up/project-name :com.wsscode.pathom3.error/node-errors}}]}
#2022-07-0511:12roklenarcicwait, wrap-attribute-error can change how errors are returned? The docs made it seem like it was for side-effects only (like logging)#2022-07-0511:14wilkerluciowhat part of the docs? lets see if we can improve it, because this point is intended for the user to modify the error#2022-07-0511:15wilkerlucioso I give a huge thing at first, so you can tune it down if you have too, but start from full data#2022-07-0511:29roklenarcicin the page https://pathom3.wsscode.com/docs/plugins#2022-07-0511:29roklenarcichttps://pathom3.wsscode.com/docs/plugins#pcrwrap-resolver-error#2022-07-0511:30wilkerluciothat's a different extension point, I actually don't see the :com.wsscode.pathom3.error/wrap-attribute-error there, need to add#2022-07-0511:30roklenarcicall the error handling samples elide the param#2022-07-0511:30roklenarcicin the outer fn#2022-07-0511:30roklenarcicand mek#2022-07-0511:30roklenarcicmake it look like it’s just for sideeffects#2022-07-0511:34wilkerluciothe resolver-error is mostly for side effects#2022-07-0511:34wilkerluciobut the wrap-attribute-error was missing, I'm adding it now#2022-07-0511:38wilkerlucioadd docs for attribute error extension point https://pathom3.wsscode.com/docs/plugins/#perrorwrap-attribute-error#2022-07-0511:38wilkerlucioadd docs for attribute error extension point https://pathom3.wsscode.com/docs/plugins/#perrorwrap-attribute-error#2022-06-2718:33sheluchinIs there any way to disable batch for a series of requests without manually modifying each resolver that might be involved?#2022-06-2719:33sheluchinI'm getting this error: > Execution error (IndexOutOfBoundsException) at com.wsscode.pathom3.connect.runner/missing-maybe-in-pending-batch?$fn (src/dev/com/wsscode/pathom3/connect/runner.cljc:367). and completely disabling batch does fix it, but I'd rather not modify the RAD plugin code if I can help it. It only happens when I send many requests in a loop. I wonder if it could be a Pathom bug?#2022-06-2816:09sheluchinUpdate: ok, so I think it was my fault and not a Pathom bug. I tracked the issue down to my handling of nil values in relations, and ultimately, some bad data in my DB. It was hard to track the bug down. Much of the time, the error includes some path like [:foo/bars 0 :bar/id] and some short accompanying message. Is there any way to get more error context in the error message to make debugging these sort of issues easier?#2022-06-2814:32mbjarlandseems this example from the pathom 3 docs does not work:
(def parallel-env
  (-> {::p.a.eql/parallel? true}
      (pci/register
        [random-dog-image
         random-cat-image
         random-pets])))
...or perhaps that's just me not getting it to work. How should I enable parallel execution assuming the above is not the way?
#2022-06-2817:38wilkerluciohard to infer just from this, can you show the complete example?#2022-06-2817:39wilkerlucioso far looks correct, but you also need to make sure to use the async runner, and remember that the async runner returns a promise (that will resolve to the final value)#2022-06-2908:32mbjarlandI think I had a problem with the namespace qualified keywords and the fact that almost none of the code snippets at don't show the (:require [ ... :as ...]). I finally found your namespace alias list at the start of the documentation set so I think that just solved my problem : )#2022-06-2909:35mbjarland@U066U8JQJ ok so now I have something I think perhaps should be working but the requests do not seem to be executed in parallel. Should this be enough to enable parallel execution:
(def env (-> {::p.a.eql/parallel? true}
             (pci/register (concat swapi/resolvers
                                   imdb/resolvers
                                   [swapi->imdb imdb->swapi]))))

(comment
 (time
  ; deeply nested query combining swapi and imdb
  @(p.a.eql/process env
                    {:swapi.person/id 1}
                    [:swapi.person/name
                     {:swapi.person/homeworld [:swapi.planet/name
                                               {:swapi.planet/residents [:swapi.person/name]}]}
                     {:swapi.person/films [:swapi.movie/title
                                           {:imdb.movie/actor-list [:imdb.person/name
                                                                    :imdb.person/id
                                                                    :imdb.person/as-character
                                                                    {:imdb.person/known-for [:imdb.movie/title :imdb.movie/year]}
                                                                    ]}]}]))
)
? I have some logging in my request function and I can see they are all being executed in sequence
#2022-06-2906:24stuartrexkingIs the documentation for mutation errors out of date? https://pathom3.wsscode.com/docs/error-handling#mutation-errors With version 2022.06.01-alpha the following example from the docs throws the exception rather than returning a map (as per the docs). With version 2021.07.10-1-alpha the map is returned.
(p.eql/process
  (pci/register
    (pco/mutation 'doit
      {}
      (fn [_ _]
        (throw (ex-info "Mutation error" {})))))
  ['(doit)])
#2022-06-2913:38wilkerlucioyes, you are right, this was before strict mode became the default#2022-06-2914:00stuartrexkingSo lenient mode will return the map?#2022-06-2914:01wilkerlucioyes, I'm just updating the docs šŸ˜‰#2022-06-2914:58wilkerluciocontents updated, thanks for the report#2022-06-3017:37hadilsI am trying to use (non-dynamic) resolvers with Datomic pull syntax. I thought it would map easily, but the problem is that if there insufficient data from the pull, then Pathom is throwing an error. I would like it to return nil or find a way to process the error. I am really stuck on this. Any help would be greatly appreciated!#2022-06-3017:43markaddlemanYour resolver will have to merge in the nil values. You might try something like this:
(merge {:output/one nil, :output/two nil ...}
       (datomic/pull ...))
#2022-06-3017:43markaddlemanIf you want to be a bit more dynamic, I think there's a way for your resolver to introspect the Pathom environment to discover its output variable.#2022-06-3017:45hadilsThanks @U2845S9KL! That is an excellent way to resolve my issue!#2022-06-3018:24roklenarcicNote that expects value that describes the desired output does not describe whether the properties needed are single map or a collection#2022-06-3018:31hadilsI need both.#2022-06-3019:31wilkerlucioanother option is to use lenient mode, which is fine with partial errors, but this choice should be made depending if you are more like "I need all or nothing" (strict mode, which is the default) or "get me whatever you can and I'll work with it" (lenient mode) https://pathom3.wsscode.com/docs/error-handling#lenient-mode#2022-06-3019:32wilkerluciothe approach of merging is fine and I also used it a bunch of times, but keep in mind that when you do that, you lose the ability to have a separate resolver for those keys from a different source, its not a bad thing nescessarily, but something to keep in mind#2022-06-3019:33hadilsI don’t know how to correctly use lenient mode. The errors jam up my middleware, because I don’t process them correctly. Do you have a code snippet for error handling#2022-06-3019:34wilkerluciolenient mode will always give a map with data, and error details for anything that failed, present at the same level (entity) that had that attr failure, the error handling here is different because you moved from "all or nothing" to partial errors, and when dealing with partial errors you have to handle errors for each attribute individually#2022-06-3019:34wilkerluciothis mode is recommended for example when doing UI's with Fulcro, given that most UI's are fine with partial data missing, but then the UI must check issues per attribute#2022-06-3019:35wilkerluciomakes sense?#2022-06-3019:35hadilsOk. I can check on the UI side (I am using re-frame). The difficulty I am having is that the errors don’t serialize properly, so my middleware crashes on it. Any suggestions for that?#2022-06-3019:36wilkerlucioare you using the latest pathom? serialization shouldn't be a problem, but if you are on latest and it is, let me know and we can dig on it#2022-06-3019:37wilkerlucio(latest is 2022.06.01-alpha)#2022-06-3019:38hadilsI am using
"2022.06.01-alpha"
I think Exceptions are not serializable. I am using muuntaja
#2022-06-3019:38wilkerluciook, gimme a sec to check something#2022-06-3019:41wilkerlucioI just tried this demo:
(pco/defresolver error []
  {:error (throw (ex-info "err" {}))})

(def env (-> {:com.wsscode.pathom3.error/lenient-mode? true}
             (pci/register [error])))

(comment
  (p.eql/process env [:error]))
#2022-06-3019:42wilkerluciowhen I look at the response, its just maps, no Exception entry, so I wonder what is going on at your end#2022-06-3019:42wilkerluciothis is what I see here:#2022-06-3019:42wilkerlucio#2022-06-3019:47hadilsI am using
(defn parser [] (p.eql/boundary-interface pathom))
#2022-06-3019:47hadilsI am not sure about the boundary interface.#2022-06-3019:48wilkerlucioshould yield the same result (tested here):
(pco/defresolver error []
  {:error (throw (ex-info "err" {}))})

(def env (-> {:com.wsscode.pathom3.error/lenient-mode? true}
             (pci/register [error])))

(def request (p.eql/boundary-interface env))

(comment
  (request [:error]))
#2022-06-3019:50hadilsOk, let me try it out and respond.#2022-06-3020:00hadilsI get the same result as you do.#2022-06-3020:02wilkerlucioso we have to figure whats going on during your other call, can you try to log it at your case and see what is getting in the output?#2022-06-3020:13hadilsIt works for this simple test. There’s is something else wrong with my muuntaja setup. Thanks @U066U8JQJ for your help.#2022-07-0214:18AthanGreetings Pathom followers, I am an x-adept in Pascal, C, Tcl, Python but now a convert to Clojure. Because it is for the brave and true šŸ™‚. OK honestly now that happened because I discovered Datalog query language and more recently Pathom-Fulcro !!! I believe that Clojure(script) together with these tools are a game changer for developers and web applications/services. Kudos to @wilkerlucio and other contributors for this fantastic library. I am keen to test and learn more about how I can apply Datalog-Pathom efficiently in my projects. For this reason I would like to share with you a https://gist.github.com/athanhat/2ccfad3786934ad41efc7c74674494b5. You may consider this as an alternative and enhanced version of the weather tutorial in Pathom3 documentation, unfortunately http://metaweather.com/api is down. I have a question about an example query I ran (see Graph Viz under the network timeline). It looks like these are possible paths that can be traversed with Pathom and with Green nodes those that were reached during the execution of the query plan, correct ? There are a couple of things I don't understand. A. Why the is-it-hot? resolver node is not connected to the other green resolver nodes in the path above ? B. Why the red resolver node I selected displays the message "Insufficient data calling resolver....", C. What are these grey resolver nodes paths and why they are repeated with exactly the same sequence e.g. there are multiple times of this (get-ip, ip-&gt;geocoords) AND (geocoords-&gt;locationkey, locationkey-&gt;currentweather)#2022-07-0511:05wilkerluciothis graph seems to point to some possible optimizations in the planner, gotta take a closer look at it#2022-07-0511:20wilkerluciobut to your questions šŸ™‚#2022-07-0511:22wilkerlucioA: the greens is what did ran, the fact is-it-hot isnt connected was just a matter of the graph execution order (and for AND branches, there is no particupar order, any path may run), so what happened is that the green path was the first one it tried, then, for the is-it-hot?, the resolvers precending it had already ran, so Pathom skipped over then (keeping than grey) and then ran is-it-hot?#2022-07-0511:24wilkerlucioB: this is a bug in Pathom Viz actually, the internal resolvers need fixing, I should get to that, just used your example and made an issue: https://github.com/wilkerlucio/pathom-viz/issues/82#2022-07-0511:25wilkerlucioC: grey means it didn't ran, and reason for repetition is because it couldn't optimize for some reason (the reason more likely to be because some situation I haven't predicted, gonna try your example and see what kind of optimizations we can add to simplify this execution graph)#2022-07-0519:26AthanOK thanks, it's more clear now. Glad I helped a bit with this example.#2022-07-0620:35wilkerluciohello @U03ET6PDHCK, I just tried to reproduce your graph, using the query:
[:geojs/city
    :geojs/region
    :geojs/country_code
    :geojs/organization_name
    :accu/temperature-val
    :accu/temperature-unit
    :accu/time-local
    :accu/day?
    :accu/weather-text
    :accu/hot?]
But I got a much simpler result, is this query that you used to generate the graph (I guess it is from the screenshot)
#2022-07-0620:35wilkerlucioif it is, a follow up question: are you using the latest version of Pathom 3?#2022-07-0620:35wilkerluciothis is what I got running it here#2022-07-0710:14AthanThis is my deps.edn configuration, I will try to execute again the query...#2022-07-0710:29AthanInteresting, same configuration, same query, same result as expected, obviously something is different in your setup. I am willing to investigate this further, let me know...#2022-07-0711:11wilkerlucio@U03ET6PDHCK you are using pathom 3 from last year, try using version 2022.06.01-alpha#2022-07-0711:12wilkerlucioto see what is latest you can check on clojars: https://clojars.org/com.wsscode/pathom3#2022-07-0711:39AthanOK fixed, IntelliJ/Cursive was not picking up the latest version. I replaced it with
com.wsscode/pathom3              {:mvn/version "2022.06.01-alpha"}
And I got it right this time
#2022-07-0507:09AthanHi, I worked a bit with your Hacker News Scraper. The https://pathom3.wsscode.com/docs/tutorials/hacker-news-scraper#read-item-details function will fail to parse item/title and item/url. Apparently Hacker News changed class name from "storylink" to "titlelink". Modify the code as follows:
(defn extract-item-from-hickory [el]
	{:hacker-news.item/age            (class-text el "age")
	 :hacker-news.item/author-name    (class-text el "hnuser")
	 :hacker-news.item/comments-count (->> (hs/select (hs/find-in-text #"comments$") el)
																				 first
																				 (find-text)
																				 (select-number))
	 :hacker-news.item/score          (select-number (class-text el "score"))
	 :hacker-news.item/id             (->> el :content first :attrs :id)
	 :hacker-news.item/rank-in-page   (select-number (class-text el "rank"))
	 :hacker-news.item/source         (class-text el "sitestr")

	 :hacker-news.item/title          (class-text el "titlelink")
	 :hacker-news.item/url            (->> (hs/select (hs/class "titlelink") el)
																				 first :attrs :href)
	 })
#2022-07-0511:04wilkerluciothanks! can you send a PR with the changes? there is an Edit this page button at the end of every page in the docs šŸ™‚#2022-07-0519:33AthanDone, PR https://github.com/wilkerlucio/pathom3-docs/pull/36#2022-07-0814:22wilkerlucio#2022-07-1318:03rodolfošŸ‘‹ Hello folks! Is there a way to generate a query from an existing sample data structure? The closest I could find is the shape-descriptor https://pathom3.wsscode.com/docs/shape-descriptor, but I wanted to have the corresponding query for it, already in EQL. Basically I have this:
{:user/name         "usuario"
 :user/billing-card {:card/number "123"}
 :user/friends      [{:user/id 1}
                     {:user/id 2}]}
And I wanted this:
[:user/name
 {:user/billing-card [:card/number]}
 {:user/friends [:user/id]}]
(Mine is a much more complex data structure with way more nested fields, hence the question)
#2022-07-1318:16sheluchin#meander or something similar might be helpful in a case like this. Not sure if Pathom itself has any helpers.#2022-07-1318:35wilkerlucio@U9V0ZDFB7 https://cljdoc.org/d/com.wsscode/pathom3/2022.07.08-alpha/api/com.wsscode.pathom3.format.eql#data-%3Equery#2022-07-1318:36sheluchinThat's awesome.#2022-07-1319:01rodolfo@U066U8JQJ wow, thanks! I was able to get it using https://gist.github.com/rodolfo42/12b02412058d3f57827a105133779da9 I just wrote, probably full of corner cases, but nice to know I can use that fn!#2022-07-1319:02wilkerluciothat one is solid, you can count on it, its been around since Pathom 2 šŸ˜‰#2022-07-1323:54rodolfoFor posterity, here is the corresponding version in Pathom 2: https://cljdoc.org/d/com.wsscode/pathom/2.4.0/api/com.wsscode.pathom.connect#data-%3Eshape#2022-07-1323:54rodolfoThanks again @U066U8JQJ#2022-07-1513:33roklenarcicIn Pathom2 env I had this:
::pc/mutation-join-globals [:tempids]
So porting to Pathom 3, should I do something?
#2022-07-1513:53wilkerluciouse the key :com.wsscode.pathom3.format.eql/map-select-include #{:tempids}#2022-07-1513:54wilkerluciothe value must be a set#2022-07-1513:55roklenarcicthanks#2022-07-2019:09λustin f(n)In Pathom 3, is there a reason why I would be able to write a nested query, but when I write a resolver that requires the same nesting structure that resolver gets a :com.wsscode.pathom3.error/attribute-unreachable error? Query:
[{:sf.resources/by-product-id
  [{:sf/resource [:resource/meta]}]}
 :resource.meta/category-set]
Heading data for the resolver that fails
(pco/defresolver resource-category-set
  [_ {resources  :sf.resources/by-product-id}]
  {::pco/input  [{:sf.resources/by-product-id
                  [{:sf/resource
                    [:resource/meta]}]}]
   ::pco/output [:resource.meta/category-set]})
#2022-07-2019:11λustin f(n):sf.resources/by-product-id doesn't directly provide :resource/meta , it instead goes through a few resolvers to get subfields. Would that be part of it?#2022-07-2110:19roklenarcicDoes querying just `
[{:sf.resources/by-product-id
  [{:sf/resource [:resource/meta]}]}]
work?
#2022-07-2220:38Ī»ustin f(n)Yes, that works#2022-07-2223:25Ī»ustin f(n)Found it. I was nesting improperly in my resolver. I think the query itself had enough info to handle my improper nested query and still return something 'sane', but the resolver choked on the incorrect nesting.#2022-07-2109:18jmayaalv#2022-07-2122:53Jakub Holý (HolyJak)In Pathom 2. why does (_parser {} [:no-such-thing :com.wsscode.pathom.core/errors]) return {} and not info about not having found :no-such-thing ? Am I missing some plugin / query / ... ? šŸ™ I see I do have p/error-handler-plugin but also (p/post-process-parser-plugin p/elide-not-found) (p/post-process-parser-plugin elide-reader-errors) . That would likely explain why I see no errors. Still, why does Pathom return {} and not nil ?#2022-07-2123:02wilkerlucioyes, the plugin elide-not-found is removing the info, if you take that out you will start seeing a not found value instead#2022-07-2123:08Jakub Holý (HolyJak)So it is by design that Pathom does not return nil here?#2022-07-2123:48wilkerlucioyes#2022-07-2214:43roklenarcicIn pathom 3 resolvers’ resolve function is it possible to access properties from the config map of the resolver?#2022-07-2214:53markaddlemanThis information is available from the environment https://pathom3.wsscode.com/docs/environment#2022-07-2214:57roklenarcicI’ve looked at env pretty print and I don’t see any data about the resolver#2022-07-2214:57roklenarcicand the ones described here don’t contain resolver either…#2022-07-2214:58markaddlemanYes, the information about the resolver is not in the docs but I think I've looked up resolver inputs and outputs through the environment#2022-07-2214:58markaddlemanI don't recall the specifics, though#2022-07-2214:58roklenarcicyes you can get inputs and outputs, but you cannot get say ::my-ns/prop that you put in resolver map#2022-07-2214:59markaddlemanAh, my apologies. I misunderstood what you are trying to do#2022-07-2214:59roklenarcicI did find a workaround though, so no worries#2022-07-2216:26Bjƶrn Ebbinghaus@U66G3SGP5 Have you tried pci/resolver-config ? https://cljdoc.org/d/com.wsscode/pathom3/2022.07.08-alpha/api/com.wsscode.pathom3.connect.indexes#resolver-config#2022-07-2216:02roklenarcicHm I am writing a dynamic resolver and I noticed that if I wrap it into a placeholder, the node-resolver-input of the dynamic resolver doesn’t get that. So {(:>/placeholder {:a/a 1}) [:dyn/attr]} doesn’t get :a/a 1 in input as expected… or am I reading this wrong?#2022-07-2217:52pithylessUnless I misunderstood, you need to get :a/a from (pco/params env) https://pathom3.wsscode.com/docs/resolvers#parameters#2022-07-2218:17roklenarcicIn my experience the params given to placeholder are given as inputs to resolver below#2022-07-2218:17roklenarcicAlso note the examples#2022-08-0218:35wilkerluciomaybe a bug, can you send a full example @U66G3SGP5?#2022-08-0218:54roklenarcichttps://gist.github.com/RokLenarcic/f56845def5c728d69389830de3f3fc94#2022-08-0219:11roklenarcic*edited the gist#2022-08-0219:13wilkerlucioon the first invoke, the behavior is correct, because you have an ident inside the placeholder, the ident makes a new entity and doesn't pull anything from the parent entity#2022-08-0219:13wilkerlucioso the context in which ::name and ::other appears are different#2022-08-0219:14roklenarcicok#2022-08-0219:17wilkerluciothe second is also what is expected, because you never get things on your resolver that you dont ask on the input, this is by design to prevent implicit dependencies#2022-08-0219:17wilkerluciobut you can use (p.ent/entity env) if you want to see the full available data#2022-08-0219:18roklenarcicok#2022-08-0219:18roklenarcicthough I would expect (pco/? to cause it to supply the data#2022-08-0219:19roklenarcicso if input is [::id (pco/? ::other)] and both are available I would expect that both are in nore-resolver-input#2022-08-0219:19wilkerluciohumm, that optional case seems a bug#2022-08-0219:19wilkerlucioit should be there if available#2022-08-0219:20wilkerluciobut I like to give a heads about something that might change and break all this, that is related to data via placeholders#2022-08-0219:20wilkerlucioI recently did some work that demonstrated to me that placeholder params as directly data might not be a good idea
#2022-08-0219:21wilkerluciobecause I see a case where I needed some specific placeholder params, that are not related to input data, and the current behavior kind forces it to happen#2022-08-0219:21wilkerlucioso I'm considering breaking changes on this#2022-08-0219:22wilkerlucioinstead maybe use some specific parameters to flow data down (like: (:>/ph {:pathom/merge-data {::foo "bar"}}))#2022-08-0219:22wilkerluciothis way Pathom allows for a better extension story on params#2022-08-0219:23roklenarcicsure#2022-07-2308:55Quentin Le GuennecHello. In Pathom 2, how can I get a join in a resolver input?#2022-07-2317:17souenzzoif you are expecting nested inputs, it is only available in pathom3 https://pathom3.wsscode.com/docs/resolvers/#nested-inputs#2022-07-2312:00sheluchinWhat is the difference between :com.wsscode.pathom3.error/attribute-missing and :com.wsscode.pathom3.error/missing-output? The https://pathom3.wsscode.com/docs/error-handling/#attribute-missing-on-output only include an entry for the first one. > This happens when the resolver completed with success, but the output didn't include the attribute.#2022-07-2512:48souenzzoAbout missing-output:
(let [a (pco/resolver `a
          {::pco/output [:a]}
          (fn [_ _]
            ;; :com.wsscode.pathom3.error/missing-output
            ;; when a resolver say that it will return :a, but dont
            {}))
      env (pci/register [a])]
  (try
    (p.eql/process env [:a])
    (catch Throwable ex
      (:com.wsscode.pathom3.error/cause (:com.wsscode.pathom3.error/error-data (ex-data ex))))))
  
#2022-07-2512:51sheluchin@U2J4FRT2T that sounds the same as "output didn't include the attribute", no? What bit of nuance am I missing?#2022-07-2512:52sheluchinIs it for when the whole output is missing?#2022-07-2512:52souenzzoI'm trying to create a :com.wsscode.pathom3.error/attribute-missing error in my repl#2022-08-0218:37wilkerluciothey work at different times, attribute missing is more broad, means attribute was expected by query, but wasn't there, this check runs after the graph is done for given entity, makes sense?#2022-08-0218:39sheluchin@U066U8JQJ so attribute-missing can happen during the graph traversal, whereas missing-output only happens at the end?#2022-08-0218:40wilkerluciohumm, I'm doing a double check reading the code, I'm starting to think one of them might not happen, because they are in the two different places, aparently for the same reason#2022-08-0218:40wilkerlucioI can't go deep check now, but I'll open an issue to clear this up#2022-08-0218:41wilkerluciohttps://github.com/wilkerlucio/pathom3/issues/149#2022-08-0218:41sheluchinOkay, thank you. I do believe I've actually encountered both errors during development, but I don't have a repro at the moment.. I could be mistaken. If it comes up again I will document it somewhere and leave a link here.#2022-07-2416:04Quentin Le GuennecIs there an equivalent of ::p/fail-fast? in pathom 3 ?#2022-07-2418:41souenzzopathom3 is "fail fast" by default https://pathom3.wsscode.com/docs/error-handling/#lenient-mode#2022-07-2420:15Quentin Le GuennecI see, thank you.#2022-07-2508:35Quentin Le GuennecI got a resolver in pathom 3 which is correctly parsed and resolved in the repl but the same resolver with the same input doesn't resolve when I parse it in the definition of a resolver. Could this be a bug?#2022-07-2512:42souenzzoCan you provide a example, like this one:
(let [;; a resolver
      a (pco/resolver `a
          {::pco/output [:a]}
          (fn [_ _]
            {:a 42}))
      env (pci/register [a])]
  ;; that is correctly parsed in parser
  (prn (p.eql/process env [:a]))
  ;; but not when I use the definition
  (prn  (a {} {})))
#2022-07-2518:46Quentin Le GuennecNope not exactly. Sorry if this is unclear. This works fine: (parser {} [{[:person/id "id"] [:a/name]}]) But somehow if I use the same in here:
(pco/defresolver test  [{:keys [parser] :as env} {:person/keys [id]}]
  {::pco/input  [:person/id]
   ::pco/output [:person/test]}
  {:person/test
   (parser env [{[:person/id id] [:a/name]}])})
It resolves to nothing. The inputs are exactly the same in both cases.
#2022-07-2518:47Quentin Le GuennecI should point out that the test resolver has ::pco/batch? true in the real code.#2022-07-2519:02souenzzo@U0111PVCS8P by running this code
(let [a (pco/resolver `a
          {::pco/output [:a]}
          (fn [{:keys [parser]} _]
            {:a (nil? parser)}))
      env (pci/register [a])]
  (p.eql/process env [:a]))
=> {:a true}
I can see that there is no :parser key in the env pathom3 do no have parser concept You should call process explicitly
(pco/defresolver test  [env {:person/keys [id]}]
  {::pco/input  [:person/id]
   ::pco/output [:person/test]}
  {:person/test (p.eql/process env [{[:person/id id] [:a/name]}])})
#2022-07-2519:23Quentin Le Guennec@U2J4FRT2T Thanks for your answer. Same behavior with p.eql/process, tho. I think parser is something fulcro injects.#2022-07-2519:26Quentin Le Guennecin the env map: :parser #function[com.wsscode.pathom3.interface.eql/process]#2022-07-2519:33souenzzo
(pco/defresolver test  [{:keys [parser] :as env} {:person/keys [id]}]
  {::pco/input  [:person/id]
   ::pco/output [:person/test]}
  (try
    (println [:start id])
    (let [return (parser env [{[:person/id id] [:a/name]}])]
      (println [:ok id return])
      {:person/test return})
    (catch Throwable ex 
      (println [:fail id ex])
      (throw ex))))
#2022-07-2519:39Quentin Le GuennecI think I’m getting more information if I remove the lenient mode injected by fulcro.#2022-07-2522:02Quentin Le GuennecI asked in the fulcro channel as it seems more fulcro than pathom related.#2022-07-2521:06roklenarcicin eql AST what is the difference between :key and :dispatch-key#2022-07-2607:03Quentin Le GuennecAre recursive calls to p.eql/process supposed to work?#2022-07-2709:10souenzzoYes#2022-07-2713:17Quentin Le GuennecOk. Should I purge the env with the current plan if I do so?#2022-07-2713:39souenzzoOh, I see. I had this cache/context issue in pathom2. By doing this, we can discover dynamically which keys pathom3 create in the process
(def keys-to-dissoc 
  (let [env (pci/register 
              [(pco/resolver `a
                 {::pco/output [:a]}
                 (fn [env _]
                   {:a (set  (keys env))}))
               (pco/mutation `b 
                 {}
                 (fn [_ _]))])
        init-ks (set (keys env))]
    (remove init-ks (:a (p.eql/process env [:a])))))
Then you can call (p.eql/process (apply dissoc env keys-to-dissoc) ...) Not an elegant solution. In theory, I think that nested process calls should be able to share indexes/caches/plans. What is the actual problem that you are having?
#2022-07-2713:41Quentin Le GuennecWell when I call process in a resolver, the output I get is different (empty) from when I get when I call that same query outside the resolver. It works when I pass the env before that resolver call.#2022-07-2706:44Quentin Le GuennecAnyone please?#2022-07-2709:31Bjƶrn EbbinghausI replied to your message in the Fulcro channel. #2022-07-2709:36Quentin Le Guennecthanks.#2022-07-2916:37sheluchinWhat is the way to use Pathom with a fully normalized SQL database that uses natural composite primary keys instead of surrogate keys (uuids)? Is it more pragmatic to just add a UUID column and use that?#2022-07-2917:34souenzzowith pathom3 placeholders https://pathom3.wsscode.com/docs/placeholders/#provide-data Should be more natural to pass multiple parameters when asking for an attribute for example
[{(:>/user {:user/username "souenzzo" :user/org-id "clojurians"})
  [:user/name
   :user/email]}]
With this query, you should be able to create a resolver that receives :user/username and :user/org-id, and lookup in the DB using the composite key.
#2022-07-3011:24roklenarcicthere is nothing stopping you from having a composite value in ident.#2022-07-3011:25roklenarcic{[:user/composite-id ["souenzzo" "clojurians]] [:user/name]}#2022-07-3011:57souenzzoYes you can do this since pathom2#2022-07-3012:09sheluchin@U2J4FRT2T thank you for the suggesting to use placeholders for this. I'll take a look at that approach. I understand that your ident value can be anything and that this makes using composite keys possible. I am wondering whether this is a good approach to take in practice instead of just using a uuid. Sometimes the composite PKs can be quite a number of fields - I think technically up to 32 or so - and while passing around the whole composite key would have technical merit, most of my pre-clojure experience is with Django, where using a UUID instead of a composite key is the standard practice and mostly works okay. There's an open 17 year old https://code.djangoproject.com/ticket/373 about it in the tracker šŸ˜„ Have you guys tried both ways? If so, did any significant tradeoffs lead you to prefer one option over the other?#2022-07-3012:12roklenarcicI mean i have never worked with such a massive amount of components in a key mostly 2 or 3#2022-07-3012:12roklenarcicAs far as software goes there really isn’t much of a difference between putting 32 keys into a placeholder or ident#2022-08-0412:01sheluchinI think there's an ergonomic/DX factor to using a large number of keys. I guess I'll try both ways and see what sticks. Thanks for the input.#2022-07-3121:21roklenarcicHow do I get to params in foreign AST? So I have a dynamic resolver, which trigger with query [{([:my/id 1] {:param/test "test"}) [:my/name]}] so query with an ident with a param on it. The foreign AST my dynamic resolver gets is {:type :root :children […. node for :my/name]} . I couldn’t find :param/test anywhere in the whole environment in my dynamic resolver resolve function#2022-08-0109:14souenzzoyou are probably asking how to access "parent" params in :my/name resolver. Well, pathom do not "support" that, but as #fulcro add the parameters in this place by default, I used to use a code like this in one of my projects:
(defn eql-api
  [req]
  (let [query (... req)
        ast (eql/query->ast query)
        root-params (-> ast
                      :children 
                      (->> (keep :params)
                        (apply merge)))
        result (parser (assoc env :root-params root-params)
                 query)]
    {:status 200
     :body   result ...}))
#2022-08-0202:17markaddlemanI'd like the input maps to resolvers be smart maps so I can pass the input to internal functions which can extract any data they need out of the environment and other resolvers. I wrote the following plugin:
(p.plugin/defplugin smart-map-plugin
  {::pcr/wrap-resolve (fn [resolver]
                        (fn [env input]
                          (let [smart-input (if (psm/smart-map? input)
                                              input
                                              (psm/smart-map (-> env
                                                                 (pcp/reset-env)
                                                                 (psm/with-error-mode ::psm/error-mode-loud))
                                                input))]
                            (resolver env smart-input))))})
Probably not surprisingly, this results in a stack overflow when I try some basic EQL queries. What should this plugin do in order for the wrapped resolver to receive a smart map as input?
#2022-08-0417:30Ben GrabowInstead of resolvers pulling their dependent data out of a smart map, why not declare the dependent attributes in the arglist of the resolver?#2022-08-0417:32markaddlemanLocality and convenience. That approach would require updating all resolvers using a function when that function's parameters change#2022-08-0417:34Ben GrabowI don't quite follow. Do you have a concrete example that we could refer to?#2022-08-0417:38Ben GrabowI'm looking at your other message here and it looks like there's a dependency cycle implied here:
(pco/defresolver id->user [{id :acme.user/id :as input}]
  (println (:acme.user/birth-year input))
  {:acme.user/name "Mark"})
(pco/defresolver name->birth-year [{name :acme.user/name}]
  {:acme.user/birth-year 1970})
:acme.user/name explicitly depends on :acme.user/id and implicitly depends on :acme.user/birth-year, but :acme.user/birth-year explicitly depends on acme.user/name. So if we have an id we can't resolve either a name or birth-year because there's no path to get one without the other.
#2022-08-0418:43markaddlemanGood point. Does it surprise you that it completes at all?#2022-08-0418:46markaddleman> I don’t quite follow. Do you have a concrete example that we could refer to? I don’t have a concrete example at hand. Suppose I have two resolvers, R1 and R2. These two resolvers call function F which takes a kw-args map as a parameter. In order to make this code function, I must duplicate the keywords from F’s parameter into the inputs of R1 and R2. If F’s parameters change, then I must update both R1 and R2. However, if I can pass a smart map into F and all of F’s parameters can be computed from the environment, then there is no code duplication and I don’t have to remember to update R1 and R2 when F changes.#2022-08-0418:47Ben GrabowWhy not make F a resolver also?#2022-08-0418:48Ben GrabowIf Pathom is a system for managing function dependencies, and F has a new dependency, then I think it makes sense to expose those dependencies to Pathom for Pathom to manage them instead.#2022-08-0418:49markaddlemanCurrently, my code is structured like that: Lots of internal functions are resolvers. This approach definitely works but introduces unnecessary abstractions. Under this approach, F needs a name and an output key. The output key is superfluous and my meager brain gets overloaded.#2022-08-0418:49Ben GrabowR1 and R2 instead of calling F directly, they can put F's return attribute in their input signature.#2022-08-0418:51markaddlemanIn addition, there are plenty of times when F only called after some condition is true. If F is a resolver, then F can be executed unnecessarily .#2022-08-0213:58markaddlemanThis little sample program works. The execution is a little surprising. There are many printlns when I would have expected a stack overflow#2022-08-0412:01sheluchinIs it okay to nest mutations? I'm presuming no, but thought I'd ask anyhow.#2022-08-0412:10souenzzopathom engine should be able to process.#2022-08-0412:10souenzzobut i don't think that it make sense for EQL's. I would classify as a "undefined behavior"#2022-08-0412:12sheluchinIs it bad form, even if technically possible? Ah, got it. Thought so. So if you want to run a bunch of mutations in series you just make a regular function that just calls p.eql/process on different mutation vectors or just use one vector with multiple mutations if there is no logic to perform between mutations, right?#2022-08-0412:14souenzzosomething like [(batch-mutation {:mutations [(create-user {}) (login-user {})]})] ?#2022-08-0412:18sheluchinI was thinking either:
(p.eql/process env [(create-user {}) (login-user {})])
or
(p.eql/process env [(create-user {})])
;; use result from first mutation somehow
(p.eql/process env [(login-user {})])
#2022-08-0412:18souenzzoI would do a single [(create-user-and-login {})] mutation#2022-08-0412:19sheluchinWhy? Then you lose the ability to use those components separately.#2022-08-0412:20souenzzoyou can keep the 3 operations#2022-08-0412:20sheluchinIf you do that you duplicate the logic from create-user and login-user, no?#2022-08-0412:22souenzzoyou can reuse at funciton level. What the frontend will do if [(create-user) (login)] fail? it will test to see if it failed at create-user or at login ? then decide if will repeat both of just one?#2022-08-0412:23souenzzocreate-user-and-login can rollback the user creation if the login fail.#2022-08-0412:24sheluchinI see your point. Hmm, defmutation is just a wrapper around defresolver, right? In that case, I guess mutations can be called as regular functions the same way resolvers can? https://pathom3.wsscode.com/docs/resolvers/#invoking-resolvers#2022-08-0412:28souenzzobeing on the server, you can even do things like this:
(defmutation create-user-and-login [ctx params]
  (let []
    (sql/with-rollback
      (p.eql/process ctx `[(create-user ~params)])
      (p.eql/process ctx `[(login ~params)]))))
But I agree with you, that calling the mutation directly should be better, to avoid huge stacktraces.
#2022-08-0412:31sheluchinYes, I guess if you use the mutation as a function you skip Pathom's tree traversal and accompanying trace in case of an error. Good point.#2022-08-0414:11wilkerlucioone thing to consider if you call the mutation directly is that you also dont get the mutation join processing (if you have a query to run over the mutation response)#2022-08-0414:16sheluchinThanks @U066U8JQJ, I hadn't considered that. Seems like a fair tradeoff to make in some cases. Good set of options here.#2022-08-0414:19wilkerlucioyeah, calling the mutation directly is really just like calling a regular fn, with a fixed interface (`env params`)#2022-08-0414:19wilkerlucioits a convenience, and if your usage makes sense, go for it šŸ™‚#2022-08-1022:41nivekuilSaw this https://github.com/Holo314/Coeffect and thought about how much more powerful pathom is. Is pathom a coeffect system? http://tomasp.net/coeffects/#2022-08-1201:31wilkerlucioI personally haven't though it that way, IME coeffects have been used to describe a data format that triggers "effects" (database updates, message passing...)#2022-08-1201:33wilkerluciolooking around I found this: https://dl.acm.org/doi/pdf/10.1145/2628136.2628160#2022-08-1323:03nivekuilI just realized something.. can fulcro's state map be replaced with a pathom smart map? With that connected to a server env, no more explicit loads needed?#2022-08-1400:08wilkerluciothe issue here is performance, fulcro state reading is a critical path and the algorithms there are tuned to make it fast, the pathom overhead here is prob big enough to hurt the user interface responsiviness#2022-08-1400:14nivekuilI thought smart map was about as fast as a normal map when cached#2022-08-1400:18wilkerlucioyou got a point, could try, the other limitation is around async, since a lot client side things require async process, but for things like full name computation it could work#2022-08-1400:49nivekuilAh right, the map accesses can't be batched like eql. Unless the interface delayed it like with the parallel resolver#2022-08-1517:12sheluchinI find debugging Pathom resolvers/mutation errors very difficult. Are there any articles or other resources that can help me?#2022-08-1517:13wilkerluciowhat is current debug flow, can you tell me what kind of issue you trying to debug?#2022-08-1517:21sheluchinFor example the issue I'm dealing with right now is in using the GraphQL GitHub integration alongside my own resolvers. I'm running into an issue where I can make a request for :github.Repository/stargazerCount, and I can make a request for :github.Repository/description, but when I make a request for both of them together like [:github.Repository/stargazerCount :github.Repository/description], the description always fails. My general approach to troubleshooting this sort of thing is to try to decompose the request into smaller parts - that's how I know that requesting the attributes individually works - and tapping inside resolvers to make sure the inputs are provided. I look at the graph in Viz as well, but at least in this particular example, I don't know how to use the visualization to solve the problem#2022-08-1518:33wilkerlucioI think understanding how to read the graph can greatly help, does the arrow and nodes in this graph make sense to you?#2022-08-1518:49sheluchinYes, it kind of does. I've read over the Planner page and the debugging page before. The color legend is straightforward. So here it looks like the bottom AND node is the problem. The black arrows only go after the orange arrows go, but the nodes that the orange arrows point at are grey, meaning they didn't execute. I guess that's why the red node is there? I don't have the clearest understanding of the picture here.#2022-08-1518:51wilkerluciothe reason they don't executed is likely to be because they already ran somewhere else (duplicated node), what I would look for in this is at the red clal to pathom-entry-dynamic-resolver, if you click on it you can see the details like: what input values it used, what AST call it made#2022-08-1518:51wilkerlucioif the inputs looks wrong for example, then I start navigating back in the graph to see what the previous resolver has done#2022-08-1519:30sheluchinIn this case, the red node is the one for description:
:com.wsscode.pathom3.connect.runner/node-resolver-input {}},
   :missing {:github.types/Repository {}},
   :required {:github.types/Repository {}}},
  :com.wsscode.pathom3.error/error-message
  "Insufficient data calling resolver 'github/pathom-entry-dynamic-resolver. Missing attrs :github.types/Repository",
The previous node is the one that gets stargazerCount:
{:com.wsscode.pathom3.connect.operation/op-name
 github/repository-ident-entry-resolver,
 :com.wsscode.pathom3.connect.planner/expects
 {:github.types/Repository {}},
 :com.wsscode.pathom3.connect.planner/input
 {:github.Repository/name {}, :github.User/login {}},
 :com.wsscode.pathom3.connect.planner/node-id 152,
 :com.wsscode.pathom3.connect.planner/node-parents #{190},
 :com.wsscode.pathom3.connect.planner/run-next 138,
 :com.wsscode.pathom3.connect.runner/node-done? true,
 :com.wsscode.pathom3.connect.runner/node-resolver-input
 {:github.Repository/name "pathom3",
  :github.User/login "wilkerlucio"},
 :com.wsscode.pathom3.connect.runner/node-resolver-output
 {:github.Repository/stargazerCount 111},
And I have an entrypoint which uses the repo name and owner:
::p.gql/root-entries-map {"repository" {"name"  ["Repository" "name"] 
                                        "owner" ["User" "login"]}     
It looks like stargazerCount is able to use the entrypoint, but the entrypoint does not get used for the subsequent :github.Repository/description call. Does that seem like the correct interpretation @U066U8JQJ?
#2022-08-1519:31wilkerlucioyeah, looks right, this could be a bug in the graphql impl, or in the dynamic resolver in general#2022-08-1519:32wilkerlucioif you can cook a demo repro I'll be glad to take a look#2022-08-1521:16sheluchinhttps://github.com/sheluchin/pathom3-graphql/commit/6f00a4d484ca0775963a945932ee5b780d256102 Thanks @U066U8JQJ. Would you like an issue created for this?#2022-08-1521:16wilkerlucioyes please, in Pathom 3 repo šŸ™‚#2022-08-1521:29sheluchinOkay, here it is https://github.com/wilkerlucio/pathom3/issues/151 Your Slack status says you're on vacation, @U066U8JQJ. This stuff can wait! Please enjoy your vacation and thank you for the help today.#2022-08-2117:04wilkerlucio@UPWHQK562 I did a fix, but I see more things to improve, if you bump pathom3-graphql to latest it should work#2022-08-2117:04wilkerlucioI add more details in the issue#2022-08-2117:16sheluchin@U066U8JQJ thanks very much. I'll test it in the next few days. Glad to hear you found more areas for improvement. The GraphQL integration stuff feels a bit like magic when everything is working. If there's any testing or intel I can provide please let me know.#2022-08-2123:32wilkerlucio@UPWHQK562 just got the new optimizations in! check what the graph looks like now šŸ™‚#2022-08-2200:40sheluchin@U066U8JQJ now that's a much cleaner picture :) looking forward to making use of this. Thanks!#2022-08-1714:42markaddlemanI’ve been thinking more and more about the ideas expressed in https://clojurians.slack.com/archives/C87NB2CFN/p1653418610858519 . I’m thinking that @wilkerlucio is right that ā€œnot everything needs a name.ā€ By that, I think he means that not everything needs to be a Pathom attribute. On the other side, one of the key benefits of Pathom is its automatic resolver composition ability: a resolver simply declares what it needs and Pathom does the mechanical work of compositing a resolver chain to produce the necessary result. The problem that I have is my application has a lot of internal-only attributes because I want to take advantage of of auto-composition but the downside is that I would prefer to use these resolvers as functions: calling them based on logic, dynamically composing them based on business logic, and dropping the output attribute (which only adds to cognitive overload), and invoking the function with additional parameters. Smart Maps come close to what I’m looking for. Using smart maps, I can write regular a Clojure function that ā€œdeclaresā€ what it needs and Pathom will compute its dependencies. The only downside is that there are plenty of times when an input to a function is optional and, if it does not directly exist in the input, I do not want Pathom to compute it. For example, it is natural to write
(defn f[{:keys [required-param optional-param] :as smart-map}]
   (if optional-param (inc required-param) required-param))
I may not want the smart-map to compute optional-param for the purposes of this function even if there are resolvers capable of computing it. Instead, if it is already in the smart-map, I’d like the smart-map to return the value but if it does not exist, I’d like the smart-map to return nil. I can implement this in the function by writing:
(def f[{:keys [required-param] :as smart-map}]
   (let [optional-param (if (contains? :smart-map optional-param) optional-param nil)]
      (if optional-param (inc required-param) required-param))))
This is not quite as nice as the first example but it works. I’m thinking of a new Pathom operator that would combine the best of both worlds and I’d like to get this channel’s thoughts before I write it. The new operator is pco/??. It behaves similarly to the pco/? operator except that pco/?? would only return a value if it is directly available. Thus, the code above could be written as:
(defn f[{:keys [required-param (pco/?? optional-param)] :as smart-map}]
   (if optional-param (inc required-param) required-param))
I have not dug into the implementation yet but I think this operator require will require a new defn-like macro. Any thoughts on this approach? How useful do you think it would be?
#2022-08-1914:09wilkerluciothis would require changes in processing and planning, the syntax thing is the easy part, optional marks are EQL params in the query, so you may add any param you need#2022-08-2115:05markaddlemanThanks. I’m going to experiment with this#2022-08-1818:04JoelWhat is a good way to test with pathom? I’m thinking using resolvers that return dummy data or generated data, and then test queries against that. Or, is there a better way to inject testing facilities?#2022-08-1910:55sheluchinYou can call resolvers as regular functions and test them as you would normal functions. https://pathom3.wsscode.com/docs/resolvers/#invoking-resolvers#2022-08-1914:09wilkerluciofor isolated tests calling the resolvers directly is a good way to go#2022-08-1914:10wilkerlucioI personally prefer more integration style tests, so you can emulate the situations your app needs, going over the whole pathom process thing#2022-08-2117:00nivekuiltrying to use pathom-viz-connector from within a container. I can curl the pathom-viz IP:8240 from inside the container, and :com.wsscode.pathom.viz.ws-connector.core/host is set to that IP, but nothing's showing up in pathom-viz. I looked at the code https://github.com/wilkerlucio/pathom-viz-connector/blob/1d2662b410f262f74da542b7743b23f1438e3adb/src/com/wsscode/pathom/viz/ws_connector/impl/http_clj.clj and it looks like host and port might be ignored in the clj impl?#2022-08-2117:11nivekuilalso is there any built-in code to do this for pathom3 https://cljdoc.org/d/com.wsscode/pathom/2.4.0/doc/pathom-developers-guide#_setting_up_the_index_explorer_resolver#2022-08-2117:11wilkerlucioPathom 3 does this automatically via the boundary interface#2022-08-2117:13wilkerluciowait, maybe not, let me check something#2022-08-2117:15wilkerlucioyup, thats right, boundary interface does it, you can copy from it if you like to do it without using the boundary context: https://github.com/wilkerlucio/pathom3/blob/1e440a61243dc1dea59e4946ebffc4ac424253f4/src/main/com/wsscode/pathom3/interface/eql.cljc#L176#2022-08-2117:16nivekuilwhat do I need on my end? I thought the boundary interface was a client api, so pathom-viz does it. I just have the parser http endpoint#2022-08-2117:17wilkerlucioyou should use the boundary interface in your http handler, so it supports the pathom API format and also include indexes (which will make index explorer available)#2022-08-2117:17nivekuilah got it#2022-08-2117:17wilkerlucioalso your endpoint needs spit transit, with the transit encoders setup#2022-08-2117:18wilkerluciothis tutorial has a sample of doing this setup: https://pathom3.wsscode.com/docs/tutorials/serverless-pathom-gcf#2022-08-2117:27nivekuilok, that was really easy to change. thanks :)#2022-08-2117:30nivekuilcan I get traces from the Query tab of pathom-viz or is that only through the connector#2022-08-2117:31wilkerlucioyou should be able to get traces too, but there is one more thing you have to do, that is expose the metadata of the entities via transit#2022-08-2117:31wilkerluciothe :transform t/write-meta on the example config tutorial#2022-08-2117:31wilkerlucio(this is a transit config, if you are using it directly you should be able to set it there)#2022-08-2117:32wilkerluciothat's because trace info comes via metadata in the response maps#2022-08-2118:26nivekuilhm, one resolver returns a response with its own metadata, where the metadata contains a function, so it can't be serialized. Is this bad practice to have metadata in my output?#2022-08-2118:27wilkerlucioideally nothing should break serialization#2022-08-2118:28wilkerlucioI made this wrapper on Transit just to get this working properly: https://github.com/wilkerlucio/transito#2022-08-2118:29wilkerluciowhat you need is to set the transit default handler to some "unknown" thing, so it can output it, I had issues in the past trying to make that strait on transit writer, so I have written my own writer that does it, maybe nowadays its fixed in Transit and you can just configure there instead of making a custom Writer#2022-08-2118:33nivekuilI might just refactor this to get rid of the metadata. The problem it's solving is setting a cookie from mutation, so the metadata is a function run by the ring handler to transform the response#2022-08-2118:35nivekuilit seems pretty easy to set the default handler now, just :default-handler (transit/write-handler (fn [ļæ±_ļæ±] "unknown") #(pr-str %)) in muuntaja's :encoder-opts#2022-08-2120:11roklenarcicDatalevin Pathom 3 Dynamic resolver https://github.com/RokLenarcic/datalevin-pathom#2022-08-2212:00Ernesto GarciaSorry if this is a basic question, but I can't figure out how to return an error response through an EQL endpoint for queries that refer to an entity id that doesn't exist. If I query for
[{[:person/id "non-existing-id"]
  [:person/id :person/name]}]
I will obtain
{[:person/id "non-existing-id"]
 {:person/id "non-existing-id"}}
as my resolver for person/name is returning nil. My Fulcro client application doesn't obtain the value for the person-name attribute, but it is believing that there is a person with id "non-existing-id".
#2022-08-2219:29wilkerluciohello @U015KAUQRFE, welcome šŸ™‚ not at all, its a difference in mindset around modeling information. in a standard entity based its common have this notion of if an entity exists or not (like: is there a row for it in the database?), but in Pathom, using attribute modeling, the question becomes a bit more complex, because there is no notion of the entity as a single thing, instead its about attribute reachability#2022-08-2219:31wilkerlucioin this world, what you can do, is define some attribute to tell you specifically that (like: :person/exists-in-db?), make an attribute to do that check (directly going to the db, or depending on something else that you always expect present for a user stored in db), makes sense?#2022-08-2308:58Ernesto GarciaThat doesn't solve the issue, because that would involve client code making sure that every request contains the additional attribute, and I guess would need to override Fulcro defaults in order to introduce code checks when receiving the response. For now, I have removed the error-handler-plugin that the Fulcro tutorial adds to the Pathom config, and I'm throwing an exception from the resolver.#2022-08-2216:14Otto Nascarellahi yall let’s say I have a resolver that depends on 2 inputs a and b and returns c (let’s say it’s the sum of those) how can I write an EQL query to express those two values given that ident is [:a 1] or [:b 2] Cannot seem to find anything on docs… cheers#2022-08-2217:37sheluchinAre a and b global resolvers that resolve without input? If so, write a resolver that takes both of those as inputs and outputs c, which is their sum, and then just request [:c]:
::pco/input [:a :b]
::pco/output [:c]
Or perhaps you're thinking about it more like parameters? In that case you pass them in using EQL params ['(:c {:a 1 :b 2})]. If :a and :b aren't global, meaning that they require some input to retrieve, you have to provide as much data as is necessary in order get them, and then you can get c. Remember that the planner has a single entry point, but attributes which can be resolved without inputs an be retrieved at any place in the graph. Does that help?
#2022-08-2218:27Otto Nascarella[{[:a 1] [:c]}] <- how do I get the :b in there in order to get :c 2#2022-08-2218:42sheluchin
(pco/defresolver b
  []
  {:b 2})

(pco/defresolver c
  [{:keys [a b]}]
  {:c (+ a b)})

(p.eql/process (pci/register [ b c])
  [{[:a 1] [:c]}])
; => {[:a 1] {:c 3}}
#2022-08-2218:43sheluchinWhen using EQL idents to provide data you are providing a single attribute to start from.#2022-08-2219:32wilkerlucioin Pathom 3, the best way is to send an entity data, so you can query strait from the root, like: (p.eql/process env {:a 1 :b 2} [:c])#2022-08-2219:33wilkerluciowhen using the boundary interface, you can also make a request like: (request {:pathom/entity {:a 1 :b 2} :pathom/tx [:c]})#2022-08-2219:34wilkerluciobut if you need to send via query (maybe to have multiple things) you can do: [{(:>/foo {:a 1 :b 2}) [:c]}], but I might break this interface because I found some usages that make this "always use placeholder params as entity data" is problematic for some use cases#2022-08-2308:25Otto Nascarellacheers!#2022-08-2307:06mdiinIf I’m starting a new project and learning about Pathom, should I use the old 2.4 version or go with the new Pathom3 lib? Have people switched to Pathom3 in production?#2022-08-2307:11mdiinI’ve started using Pathom2 already, but ran into the need to have the parser resolve mutation params. I saw that Pathom3 has a plugin for doing just that, so that pulls in the direction of Pathom3.#2022-08-2308:17danierouxWe use pathom3 in production#2022-08-2308:44jmayaalvdefinitely pathom3.#2022-08-2308:46Tom H.We also use Pathom 3 in prod šŸ˜€#2022-08-2309:33mdiinAlright, it looks like I’m making the switch to Pathom3. Thanks, I appreciate the feedback. šŸ˜„#2022-08-2505:03wilkerlucio#2022-08-2505:44HukkaI must be missing something. To my eyes, having either ab1 or ab2 would be sufficient, but both c1 and c2 would not be.#2022-08-2505:48wilkerlucioman, I think you are right, that connection should be an OR :man-facepalming:#2022-08-2505:45Hukkahttps://blog.wsscode.com/pathom/v2/pathom/2.2.0/connect/exploration.html demo doesn't seem to find the https://blog.wsscode.com/pathom/v2/assets/js/book/main.js anymore#2022-08-2512:41wilkerlucioI'm noticing its breaking to compile when I try locally, gonna check on it later#2022-08-2514:59nivekuilI just realized there's paredit bindings (ctrl+right/left) in pathom-viz :O#2022-08-2515:02wilkerlucioits actually parinfer there šŸ˜‰#2022-08-2515:02pieterbreedWhat is the guidance for mutations that need ambient information? I have a system that up to now has only had resolvers. In the stack, the user-id was added to all inputs. Now that I created the first mutation I need that user-id too so that I know what user to perform the mutation against. What would be the recommended way to access this info? I can think of 1) add to the environment 2) add to the parameters (which would be awkward since the user-id is not known at the call-site) 3)??#2022-08-2515:03wilkerluciothe common thing is to send as a param to the mutation from the client#2022-08-2515:04wilkerluciosomething like: (user/update {:user/id 1 :user/name "New Name"})#2022-08-2515:05pieterbreedthank you. One last question. Are db connections inputs or do they belong to the environment?#2022-08-2515:06pieterbreed(or neither)#2022-08-2515:15sheluchinTypically you have connection information in your config, which you pass along in your environment.#2022-08-2518:46donovanmcgillenHi. I'm trying (and failing) to find an example of wrap-mutation-error. We use wrap-resolver-error to log and return a map (so we don't get transit marhsalling exceptions) whenever a resolver throws an exception (which shouldn't normally happen but it certainly does during development!), and I want to do the equivalent in mutations. The resolver one uses the supplied mark-node-error fn that it is given, but I'm unsure what I'm supposed to do with mutations. The plugin does get called because my logging is happening, but I'm still getting an exception telling me that the original exception couldn't be marhsalled because I'm not sure how to replace it with my error map. Any pointers? Thanks!#2022-08-2600:07wilkerluciohello, I did look and this entry point is broken, but easy thing to fix, I'm already doing some work on Pathom now, I can add that to the list#2022-08-2608:01donovanmcgillenNice one, thanks for that šŸ‘#2022-08-2608:13donovanmcgillenOh I see you've literally already fixed it and made a release - what service! Working as expected for me now, many thanks for that#2022-08-2602:09wilkerlucio#2022-08-2602:12wilkerluciohello folks, I made a release yesterday that was broken, since then I wanna be more careful with the new optimisations, they are specially useful if you are using dynamic resolvers, they may considerably reduce the number of dynamic calls. but even if you are not using dynamic resolvers and like to help mature those optimisations, if you have some large graph, I really appreciate if you can test your queries in it using the new optimisations, and report any problem you may find šŸ™ if you like to help testing, bump to latest and add :com.wsscode.pathom3.connect.planner/experimental-branch-optimizations true to your env, this flag will enable the new branch optimisations#2022-08-2607:08HukkaI haven't really used pathom viz. Should I be reading these graphs differently than just normal boolean logic? In the left side, the AND node requires either ab1 or ab2 via the top two branches. But on the right side, they aren't; any leaf node will do.#2022-08-2613:36wilkerluciothis graph is an execution graph, its tunned to minimize the work of the runner, the runner will walk this graph from the root, executing the nodes on the way, its important to know the difference between the arrows, the oranges are the branches and run first, the black is "run next", and will run only after all the orange ones run (including their children), to understand whats happening its good to understand the initial query, and see what the graph is doing, the query was: [:a :b :c], where there is 2 resolvers with output [:a :b], and two resolvers with output [:c] (both depending on [:a :b]). considering that, the graph on the left has and AND with 3 branches, one for :a, one for :b, and one for :c, what the optimizations is doing is checking that, for example, :a and :b have the same exact nodes, so we dont need that repetition, and we can merge those. similar, for the :c computation, both OR nodes have the same duplicated :a and :b requirement, there it needs to merge also the thing that runs after each of the OR paths (that lead to c1 and c2), in the end it generates the graph with the minimum amount of nodes, because both ab1 and ab2 are able to realize both :a and :b, same for :c, we only need one of c1 or c2 to get it. makes sense?#2022-08-2614:49HukkaTook a couple of tries, but I think I do#2022-08-2615:15wilkerluciosorry, all orange must run for AND nodes, in case or OR node, each orange arrow is a possible path#2022-08-2616:25HukkaYeah, the OR and AND are over the orange inputs, resulting to all black outputs#2022-08-2713:53sheluchinIt looks like the build for v2022.08.26-alpha failed and I'm also getting some Pathom errors when using RAD reports. It's complaining about Insufficient data calling resolver even when the data is obviously there. This is without enabling the experimental stuff.#2022-08-2713:54wilkerluciohello @UPWHQK562, can you make some repro that I can try? also, have you checked the plan, it looks an error in planner or runner?#2022-08-2715:10sheluchinGood morning @U066U8JQJ.
(comment ; pathom3 v2022.07.08-alpha
  (p.eql/process {}
                 {:a :_}
                 ['({[:a 42]
                     [:a
                      {[:x 99] [:x]}]})]))
=> {[:a 42] {:a 42
             [:x 99] {:x 99}}}

(comment ; pathom3 v2022.08.26-alpha
  (p.eql/process {}
                 {:a :_}
                 ['({[:a 42]
                     [:a
                      {[:x 99] [:x]}]})]))
=> ExceptionInfo
Execution error (ExceptionInfo) at com.wsscode.pathom3.connect.runner/processor-exception (runner.cljc:901).
Graph execution failed: Required attributes missing: [:a] at path [[:a 42]]
#2022-08-2715:12sheluchinThe shape is just repro of how RAD report queries are shaped.. something like this:
['({[:item/id 1]                              
    [:item/id                                 
     {[:com.fulcrologic.rad.report/id         
       :com.example.ui.reports/ItemsReport]   
      [:com.fulcrologic.rad.report/id]}]})])) 
#2022-08-2723:42wilkerlucio@UPWHQK562 thanks for the report, the issue was related to change to support ::wrap-merge-attribute (not related to planner changes) with idents, fixing it now#2022-08-2723:49wilkerlucio@UPWHQK562 can you please try sha 9fd82ed868303f94b385ba7411b3107c4a76895d and confirm it works in your scenario?#2022-08-2910:47sheluchin@U066U8JQJ Yep, looks like that fix solves the issue. No more errors in the REPL. Thank you.#2022-08-2912:23wilkerlucio@UPWHQK562 fix released: https://clojurians.slack.com/archives/C015AL9QYH1/p1661775772494209#2022-08-2602:12wilkerluciohello folks, I made a release yesterday that was broken, since then I wanna be more careful with the new optimisations, they are specially useful if you are using dynamic resolvers, they may considerably reduce the number of dynamic calls. but even if you are not using dynamic resolvers and like to help mature those optimisations, if you have some large graph, I really appreciate if you can test your queries in it using the new optimisations, and report any problem you may find šŸ™ if you like to help testing, bump to latest and add :com.wsscode.pathom3.connect.planner/experimental-branch-optimizations true to your env, this flag will enable the new branch optimisations#2022-08-2822:44Mark WardleHi all. I am running in lenient mode, but a missing attribute is having an effect on attributes that do exist. Here I am searching for a patient using a pseudonym. It works. But if I add an attribute that is designed only to be used in my user interface, all attributes have an error. I will rename the front-end component to be :ui/xxxx so that Fulcro filters it out from server side resolution, but I thought my server side code would fail gracefully for attributes not found?#2022-08-2822:52wilkerluciowhat version are you using mark?#2022-08-2822:52wilkerluciothe release from yesterday has an issue that might affect idents in mid query#2022-08-2822:53wilkerlucioif thats the case you can try the commit https://clojurians.slack.com/archives/C87NB2CFN/p1661644164969929?thread_ts=1661479753.266449&amp;channel=C87NB2CFN&amp;message_ts=1661644164.969929#2022-08-2907:39Mark WardleAmazing. I can confirm latest commit (9fd82ed868303f94b385ba7411b3107c4a76895d) fixes the issue.#2022-08-2907:42Mark WardleTo be clear, I was using the 2022.08.26-alpha but I had made some swingeing changes in code as well as upgrading my deps, so had thought I had caused it.#2022-08-2912:23wilkerluciofix released: https://clojurians.slack.com/archives/C015AL9QYH1/p1661775772494209#2022-08-2912:22wilkerlucio#2022-08-2918:57shanešŸ‘‹ is their an official way to get the ast/params in pathom3? Right now I'm looking in planner/graph but I'm not sure if that is right:
-> env
      :com.wsscode.pathom3.connect.planner/graph
      :com.wsscode.pathom3.connect.planner/index-ast
      :my-lookup-field
      :params
#2022-08-2919:21wilkerluciocheck pco/params fn#2022-08-2919:47shanethanks! not sure how I missed that - I was looking at batch resolvers all weekend and params were right next door.#2022-09-0208:54Ernesto GarciaHi all. When obtaining an error from a Pathom 3 boundary interface used in a Ring server, how do I know whether to raise a 400 or a 500 error?#2022-09-0722:52wilkerlucioif you use strict mode that might make sense to have something, but Pathom has no oppinion on that, since is also up to you to handle http response errors on the other end, currently there is not standards for it#2022-09-0608:43Ernesto GarciaFor instance, I have obtained an error with error-data being nil, which I didn't expect:
{:com.wsscode.pathom3.error/error-message "Don't know how to create ISeq from: clojure.lang.Symbol", :com.wsscode.pathom3.error/error-data nil, :com.wsscode.pathom3.error/error-stack "java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol\n\tat clojure.lang.RT.seqFrom(RT.java:557)\n\tat clojure.lang.RT.seq(RT.java:537)\n\tat clojure.lang.APersistentMap.cons(APersistentMap.java:40)\n\tat clojure.lang.RT.conj(RT.java:677)\n..."}
#2022-09-0609:28Ernesto GarciaPathom-relevant part of the stack trace, btw:
at clojure.core$update_in.invokeStatic(core.clj:6175)
	at clojure.core$update_in.doInvoke(core.clj:6161)
	at clojure.lang.RestFn.invoke(RestFn.java:467)
	at edn_query_language.core$call__GT_ast.invokeStatic(core.cljc:154)
	at edn_query_language.core$call__GT_ast.invoke(core.cljc:151)
	at edn_query_language.core$expr__GT_ast.invokeStatic(core.cljc:207)
	at edn_query_language.core$expr__GT_ast.invoke(core.cljc:199)
	at clojure.core$map$fn__5880$fn__5881.invoke(core.clj:2746)
	at clojure.lang.PersistentVector.reduce(PersistentVector.java:343)
	at clojure.core$transduce.invokeStatic(core.clj:6885)
	at clojure.core$into.invokeStatic(core.clj:6901)
	at clojure.core$into.invoke(core.clj:6889)
	at edn_query_language.core$query__GT_ast.invokeStatic(core.cljc:165)
	at edn_query_language.core$query__GT_ast.invoke(core.cljc:158)
	at com.wsscode.pathom3.interface.eql$process.invokeStatic(eql.cljc:67)
	at com.wsscode.pathom3.interface.eql$process.invoke(eql.cljc:35)
	at com.wsscode.pathom3.interface.eql$boundary_interface$boundary_interface_internal__54301.invoke(eql.cljc:192)
	at com.wsscode.pathom3.interface.eql$boundary_interface$boundary_interface_internal__54301.invoke(eql.cljc:196)
#2022-09-0722:53wilkerluciodo you believe this is a bug in pathom? im not sure from that stack, if so, please send a repro and I'll be happy to look into it :)#2022-09-0915:21Ernesto GarciaI think the error-data obtained from boundary-interface should never be nil, and boundary-interface should specify how to distinguish when there has been an error in the EQL query, or an exception thrown by resolvers, or an exception in the Pathom engine. I will look into how to reproduce the error above. I think it was just an incorrect EQL txn.#2022-09-0915:21Ernesto GarciaThanks for your response.#2022-09-0610:07Thomas MoermanHi gang, a bit of a long-shot question. I've lately been thinking and reading about hexagonal/[diplomat](https://youtu.be/ct5aWqhHARs) architectural patterns and was ruminating on how to marry this with Pathom mutations. The #interceptors approach (cfr Sieppari and Pedestal) offers a nice way of composing functionality using interceptor operators on a context map. Now my question is: when using Fulcro/Pathom, how could I break up a classic pathom mutation into an interceptor chain or part of an interceptor chain? Roughly the interceptor chain would consist of: :enter phase => [param coercion] -> [param resolution (using pathom)] -> [do permission checks] -> [pure functional business logic operation on ctx map (aka mutation body)] -> ["sync" side-effects (e.g. db transactions)] -> ["async" side effect (e.g. posting messages to queues)] :leave phase => [resolving eql mutation join] -> [redaction of eql results in function of authorization policy rules] -> [error response handling] The two steps in bold are what typical processing of a pathom mutation entails, but in the above schematic they are broken up into two discrete interceptor phases. I'm thinking in the direction of having an interceptor at the start of the chain the inspects the eql AST and dynamicall injects the interceptor building blocks in the chain, based on a data representation that describe pathom mutations (name, params and body) as well as the interceptor chains involved. The main objective of this approach would be to increase the compositional nature of mutations in a Fulcro/Pathom app. Thoughts, comments, ideas?#2022-09-0614:09Thomas MoermanThe pathom plugin systems might be a good starting point for this. It's not explicitly mentioned in the Pathom docs, but the https://pathom3.wsscode.com/docs/plugins#adding-plugins looks exactly like an interceptor-based system.#2022-09-0618:06lgesslerAlso worth looking at ::pco/transform https://pathom3.wsscode.com/docs/resolvers/#resolver-transform#2022-09-0723:00dvingo@U052A8RUT I landed on this exact pattern after learning pedestal https://github.com/dvingo/my-clj-utils/blob/99d3820b1b2bc77ba33c792b493017645786b947/src/main/dv/pathom.clj#L187 I made this helper some time ago, I have a more recent version in a project that uses sieppari instead because it has cljc support#2022-09-0723:00dvingothe pathom context substitutes exactly for the pedestal context in a http request lifecycle#2022-09-0723:03wilkerluciohello @U052A8RUT, as you found out, the plugins is the right way to handle that generically for a pathom system, also as @U49U72C4V pointed out, the ::pco/transform is another way to extract this kind of behavior, but instead of applying it globally, its uses are per mutation case.#2022-09-0723:06dvingoand here is the sieppari version https://gist.github.com/dvingo/95e3dc2be3d7d1629079f459b9086a1e#2022-09-0723:15dvingonote that sieppari is apparently being sunset, but this https://github.com/exoscale/interceptor is a good alternative#2022-09-0810:00Thomas Moerman@U051V5LLP oh is it? Good to know, thanks. Thanks all for sharing your perspective. metal#2022-09-1419:09Thomas MoermanšŸ§ŖšŸ”¬:male-scientist: 🧫 I cooked up something to illustrate my idea: https://gist.github.com/tmoerman/c4c863c00e5364932ad209f921d9e7d2 Instead of running interceptors in a mutation body, the eql-processor becomes a (2-phase) interceptor itself! I used 2 Pathom plugins, each of which is added to the env to modify the behavior of p.eql/process: one for the :enter phase of an interceptor, one for the :leave phase. This means the p.eql/process fn is run twice. In :enter the result of the mutation step is captured in acc atom, the mutation join is stripped off the ast. In :leave the captured result is used instead of executing the mutation. With some more plumbing, I would then only allow pure functions in the mutation body, capturing transactions and other side-effects in the interceptor context map, for other downstream interceptors to pick up and apply. When the :leave phase of the eql-processing interceptor is reached, the side effects have already been done and the mutation join is sure to resolve data in function of an updated DB. Not sure this approach will turn out to be more useful than running an interceptor stack in mutation bodies, but I had a good time learning about pathom plugins šŸ˜‰ The plugin system is very cool @U066U8JQJ! What do you guys think? Cheers!#2022-09-0720:54JoelI already use malli/jsonista from metosin. I was wondering if there is some clever way to map the ingested json data to clojure/pathom attributes.#2022-09-0723:04wilkerluciohello Joel, how would you like to make such an integration? can you detail one specific case you think those could be connected?#2022-09-0723:08JoelIt seemed like if I was already describing the incoming map with Malli, that potentially I could be describing the parts that pathom can use to fetch further data, that’s all. I’ve barely gotten my hands dirty with pathom, so not sure if that makes sense.#2022-09-0723:09wilkerlucioso, you have for example an external endpoint call, for which you have a malli description of the input and response for that endpoint, and you would like to use that to generate a resolver, is it like this?#2022-09-0723:10JoelNo, actually I’m working on the back-end and have an incoming request where the data I’ll be using to invoke other data endpoints using pathom.#2022-09-0723:11wilkerlucionot sure if I get that, can you send a code example?#2022-09-0723:12Joelso it’s sort of like i want to pipe the data from the request into the data that pathom knows about to fetch further info.#2022-09-0723:13JoelSure. I’ll see what makes sense.#2022-09-0723:21JoelSo if i have a simple malli validator
(def request-schema
  (m/schema [:map 
             [:employee_id string?]]))
And employee_id is something useful to fetch data with. Obviously I can grab it out of the map, but was thinking maybe there’s a way I could describe the schema with info towards identifying the info for pathom, eg, as the :employee/id field. I’ll be getting many such various requests with one or more fields that might be useful.
#2022-09-0723:22JoelI imagine it means adding data to the schema and then walking it to grab out the interesting fields to pass to pathom.#2022-09-0723:22JoelI think malli has some arbitrary property/attributes that can be embedded in the schema.#2022-09-0723:23wilkerlucioyeah, the name translation seems to be problem to deal with, you will need to define conventions or have some manual translation maps#2022-09-0723:23Joelor maybe i just make a request-resolver?#2022-09-0723:23wilkerluciothis is similar to the problem of mapping something like OpenAPI schemas, where they have all endpoints declared, but it can be tricky to turn the unqualified names into qualified ones to satisfy pathom#2022-09-0910:50sheluchinIs there anywhere I can see examples of high-level/integration style of testing?#2022-09-0914:20wilkerlucionot much out there, but the idea is simple, is to run tests over something like p.eql/process, and check for the results#2022-09-0914:21wilkerluciosomething like:#2022-09-0914:21wilkerlucio
(deftest load-user-test
  (is (= (p.eql/process (assoc env :fake-db ...)
           {:acme.user/id 1}
           [:acme.user/name
            :acme.user/email])
         {:acme.user/name  "User Name"
          :acme.user/email "
#2022-09-0914:41sheluchin@U066U8JQJ and this the approach you prefer with Pathom? Same thing for testing mutations? Do you also test underlying functions outside of the Pathom context?#2022-09-0914:43wilkerluciodepends, I think this is the kind of test I want to make sure I'm not breaking basic things, if a resolver is complex enough I might also have some unit tests for it#2022-09-0914:43wilkerluciomutations in general are simpler in terms of testing because they are one to one mapping (one mutation always calls only a single function)#2022-09-0914:44wilkerluciobut for instance, resolvers/mutations that simply forward a call to some HTTP server, I tend to don't have unit test, I make sure its working in the REPL and let it be#2022-09-0917:19sheluchinI've been over-relying on the kind of REPL-driven testing and in this particular project haven't got enough automated tests for the CI to be valuable yet. Sometimes it feels like a lazy way to do it, considering REPL tests can be copied over pretty easily and re-used in your pipeline. I guess you also use guardrails extensively? There's no >defn-like thing that can be used to instrument resolvers, right?#2022-09-0917:22wilkerlucioyou can easely write a plugin to do it, as long as you have the specs for the attributes it should be relatively easy to check then in, and also check the output#2022-09-0917:29sheluchinHmm, yeah, that sounds doable. Interesting idea. I think there's some more potential that can be unlocked if you use RAD and co-locate specs to RAD attributes. I'm going to give this some thought and maybe loop back to the idea on a future iteration. I'm going to start with some simple integration tests and add unit tests around the tricky bits - work my way inwards. Thanks for the explanation and advice, @U066U8JQJ.#2022-09-0918:02nivekuilwhat's the difference between setting ::pco/cache-store nil and ::pco/cache? false#2022-09-1117:49wilkerlucioa nil cache store will use the default one, and it still gonna cache#2022-09-1117:49wilkerluciocache false will prevent any caching from pathom side on that resolver#2022-09-1117:57nivekuilI see, I don't use default resolver cache so I guess it's the same effect for me#2022-09-1117:58wilkerluciothe default is already there, its provided by pathom#2022-09-1117:58wilkerlucioif you keep it just nil, it is going to be cached there#2022-09-1117:58wilkerluciounless you explicitally removed the default cache (you can do it setting ::pcr/resolver-cache* nil in your env)#2022-09-1117:59nivekuilI didn't know that actually.. how does it cache? ttl? is it just an atom?#2022-09-1117:59wilkerlucioits a simple atom, and the cache lives only during a single process (a call to p.eql/process), every request gets a new one#2022-09-1117:59wilkerluciothis is avoid repeated calls to the same resolver with the same input in the same process#2022-09-1118:00nivekuilah, I knew that, don't even consider it caching since performance would be terrible without it#2022-09-1118:00nivekuilI might replace that with caffiene then, probably a little faster#2022-09-1118:00wilkerlucioit is, for example, if you have a resolver that does some random computation, you might want different values at difference places in the result, so caching compromises this case#2022-09-1118:01nivekuilinteresting, I guess all my resolvers are pure so I never thought about it#2022-09-1118:02wilkerluciocurious about your results with caffiene#2022-09-1118:02wilkerlucioyou can replace the default cache store with any custom one you want, using that same ::pcr/resolver-cache* key in the env#2022-09-1118:04nivekuilI'm probably not going to benchmark it but this is my code for caffeine (with cloffeine)
(defrecord CaffeineCache [cache-atom]   p.cache/CacheStore   (-cache-lookup-or-miss [_ cache-key f]     (when cache-key ;; TODO remove when pathom stops trying to cache failed resolvers       (or (cache/get-if-present cache-atom cache-key)         (do (cache/put! cache-atom cache-key (or (f) {}))             (cache/get-if-present cache-atom cache-key)))))   (-cache-find [_ cache-key]     (some->> (cache/get-if-present cache-atom cache-key) (clojure.lang.MapEntry/create cache-key))))  (defn make-cache [name opts]   (let [c (cache/make-cache (merge {:recordStats true}                                    opts))]     (metrics/add-cache! name c)     (->CaffeineCache c)))
#2022-09-1118:04nivekuilI really like the prometheus metrics integration#2022-09-1121:12nivekuilso according to https://pathom3.wsscode.com/docs/cache/#eql-process the runner is creating/discarding the cache. where is this happening? and how? there isn't even a method on the protocol to evict/invalidate the cache store#2022-09-1117:43lilactownI'm trying out pathom3 on the front end using CLJS. works great! I noticed that my bundle is including expound. for a simple app, it's actually quite a bit of code. is there a way that we could elide it in production?#2022-09-1117:43lilactown#2022-09-1117:44lilactownhmm I also see transit-js in there which my app doesn't use. seems to be pulled in by pathom according to clj -Stree#2022-09-1117:45lilactowncould be wrong, looks like it's also pulled in by lambdaisland/fetch#2022-09-1117:52wilkerluciohello @lilactown, yeah, expound is because of guardrails, but I understand the concern, I'll see what I can do to get that out, transit is there, but just to give the user the data types mapping, if you don't use it should be removed in advanced compilation, I'll have a look on that too#2022-09-1117:53lilactownI wouldn't spend too much time on transit, I opened an issue on lambdaisland's lib https://github.com/lambdaisland/fetch/issues/20#2022-09-1118:04lilactownconfirmed that without that lib, transit-js isn't in my bundle - i.e. pathom doesn't include it šŸ™‚#2022-09-1118:06lilactowndoes guardrails really save a lot of headache while developing pathom? I've never used it#2022-09-1118:07wilkerlucioI like it, a got a couple bugs that were clearer to identify due to the guardrails checks, I also like that when I read a function declared with it, I can nav though the specs if I forget what a data type is expected to be#2022-09-1118:10wilkerlucioalso minor, but the >def to define a spec accepts a docstring (which goes nowhere, but good to read with the code), for example:
(>def ::choose-path
  "A function to determine which path to take when picking a path for a OR node."
  fn?)
#2022-09-1118:12lilactownthose do seem like nice things#2022-09-1118:29lilactownI'd also like to remove the dependency on clojure.spec.alpha, as that also is about 8% of the bundle size#2022-09-1118:29lilactownbut that seems deeply integrated into the pathom lib#2022-09-1118:30wilkerluciokind of, but maybe due to guardrails it might be possible to get shaved out#2022-09-1118:30wilkerlucioI'm working on a fork of guardrails, that will just do nothing related to specs, so it may be able to just avoid outputting anything, so advanced compilation might be able to shave it out#2022-10-0818:27mauricio.szaboIf you can remove guardrails, it may be easier to make Pathom work on other environments, like Clojerl, ClojureDart, and maybe even Cherry... That could be quite interesting wow#2022-09-1118:31wilkerluciothere is some places where its used by the library, but those are in the macros, so they dont need to be out I think, not sure#2022-10-0818:27mauricio.szaboIf you can remove guardrails, it may be easier to make Pathom work on other environments, like Clojerl, ClojureDart, and maybe even Cherry... That could be quite interesting wow#2022-09-1203:07wilkerlucio@lilactown try this, first add a deps alias:
{:guardrails-stubs
  {:override-deps
   {com.fulcrologic/guardrails {:git/url ""
                                :git/sha "063451653a8ab656b436b0258cdb438e7ffac4c6"}}}}
The do the advanced compilation with it, it should remove expound and clojure spec, this is my test result here:
#2022-09-1214:20dvingoyou should be able to use ns-alias for the release build to eliminate those https://github.com/fulcrologic/guardrails#dead-code-elimination
{:build-options
    {:ns-aliases
     {com.fulcrologic.guardrails.core com.fulcrologic.guardrails.noop}}
but yea, would be nice not to have to do that
#2022-09-1219:50wilkerlucio@lilactown this actually looks the best option to me#2022-09-1222:13lilactowni'll try this. thanks!#2022-09-1216:12Gonzalo Rafael AcostaHello Pathom and Wilker Lucio, congrats for this great project. Sorry for disturbing your vacations. I think I encountered an issue if I am not mistaken (it's my first experience with the tool) when trying to run a graphql resolver in the browser. I set up a little project to demonstrate the issue: https://github.com/revert-finance/pathom-graphql-test I am getting an exception when trying to run what I think is a valid query https://github.com/revert-finance/pathom-graphql-test/blob/main/src/revert/pathom_graphql_test.cljs#L53 This is the message: Graph execution failed: Required attributes missing: [:univ3.Token/symbol] at path [:staker.Query/incentives 0] Things to notice: • This query also fails https://github.com/revert-finance/pathom-graphql-test/blob/main/src/revert/pathom_graphql_test.cljs#L70 with Graph execution failed: Required attributes missing: [:staker.Incentive/contract] at path [] • Same queries work in clojure https://github.com/revert-finance/pathom-graphql-test/blob/main/src/revert/pathom_graphql_test.clj (is it a setup issue of my project in cljs?) Thanks in advance#2022-09-1216:23wilkerluciothanks for the report Gonzalo! I'll have a look at it, one question, when you say it runs on clj, you mean you have the same setup, both in clj and cljs, and it works in one but not the other? or are you requesting your graph made in CLJ from CLJS?#2022-09-1216:30Gonzalo Rafael AcostaHi Lucio, thanks for the quick reply. I want to run the resolver directly in the browser that's my goal without any backend dependency. What I did to test my setup and check I was not messing things up is to run the same resolver in clj, with the conclusion that in clj works but not in cljs. You can see the two files in here https://github.com/revert-finance/pathom-graphql-test/tree/main/src/revert#2022-09-1216:31wilkerluciothanks, thats fascinating, I like to dig on it and understand whats going on šŸ™#2022-09-1216:32wilkerluciothanks for making the repro, it really helps#2022-09-1216:32wilkerluciowill I be able to hit those apis? are they public/free?#2022-09-1216:32Gonzalo Rafael Acostayes they are public#2022-09-1216:32wilkerlucioawesome#2022-09-1216:33Gonzalo Rafael AcostaThank you!#2022-09-1220:18wilkerluciofound, was an oversight around dealing with async responses when starting the query from an ident request, bump pathom3-graphql to 2022.09.12-alpha and you should be fine ;)_#2022-09-1220:18Gonzalo Rafael Acostatks!#2022-09-1220:19wilkerlucioI also notice you are using an outdated version of Pathom 3#2022-09-1220:19wilkerlucioI recommend that you bump to latest 2022.08.29-alpha#2022-09-1220:20Gonzalo Rafael Acostawill do that thanks so much again for the quick response#2022-09-1220:20wilkerlucioalso, since you are in some dynamic resolver land, you may be able to take advantage of some experimental optimizations in the planner, if you see that Pathom is making more requests than you expect, try setting :com.wsscode.pathom3.connect.planner/experimental-branch-optimizations true in your env, to try the experimental optimizations
#2022-09-1220:22Gonzalo Rafael Acostaamazing thanks, will definitely play with it#2022-09-1220:25Gonzalo Rafael Acostajust checked and it's working šŸ™Œ#2022-09-1311:49sheluchinI've been wondering if anyone's applying Pathom to The Graph. Seems like an interesting pairing.#2022-09-1320:51Jakub Holý (HolyJak)Update https://github.com/wilkerlucio/pathom3/issues/156 Hello! I am clearly missing something obvious here. I am sending the mutation
[{(com.example.client.mutations/create-random-thing
   {:tmpid #fulcro/tempid["08fd736d-d3cc-4270-9980-08534c8bd7a2"]})
  [:com.wsscode.pathom3.connect.runner/attribute-errors]}]
and getting back
{com.example.client.mutations/create-random-thing
 {:com.wsscode.pathom3.connect.runner/attribute-errors
  {:com.wsscode.pathom3.connect.runner/attribute-errors
   {:com.wsscode.pathom3.error/cause
    :com.wsscode.pathom3.error/attribute-unreachable}}}}
despite having defined
(pco/defmutation create-random-thing [env {:keys [tmpid] :as params}]
  {::pco/op-name 'com.example.client.mutations/create-random-thing
   ;::pco/params [:tempid]
   ::pco/output [:tempids]}
  (println "SERVER: Simulate creating a new thing with real DB id 123" tmpid)
  {:tempids {tmpid 123}})
which I see is called from the println. What am I doing wrong? šŸ™
#2022-09-1321:06wilkerluciothe issue here is that attribute-errors come after processing, so they are not something you ask in the query (in the sense that the planner will consider something missing), that was the thinking, but you are pointing a case that makes sense, the client will ask for it, as a hack I believe if you just add an (pbir/constantly-resolver ::pcr/attribute-errors nil) it might work, but its no good really, can you please open an issue for that?#2022-09-1321:09wilkerluciooh, sorry, there is an issue already :man-facepalming:#2022-09-1321:09wilkerluciothanks for it šŸ™‚#2022-09-1321:32wilkerluciodoing some experiments#2022-09-1321:36wilkerlucioI think the add of the constantly resolve is a fine solution, what I'll do is make that automatic on lenient mode, since there is no point to it on strict mode#2022-09-1321:36wilkerlucioin case you wanna replicate, this is the extra resolver that fixes it:#2022-09-1321:36wilkerlucio
(-> (pbir/constantly-resolver :com.wsscode.pathom3.connect.runner/attribute-errors {})
                         (pco/update-config assoc ::pco/op-name (symbol "com.wsscode.pathom3.connect.runner" (str (gensym "attribute-errors")))))
#2022-09-1321:36wilkerlucio(I'm doing random name generation just because in case of graph integration I can't have resolver names colliding)#2022-09-1322:05Jakub Holý (HolyJak)Awesome, thx a lot!#2022-09-1422:48wilkerluciofixed in main#2022-09-1415:04njjHello there, does anyone have any insight as to why this first defresolver works but the 2nd ends up in an unread? Working:
(pc/defresolver process-batch [{:keys [ast]} _params]
  {::pc/input #{}
   ::pc/output [:batch-jobs/process-batch]}
  {:batch-jobs/process-batch (auth/batch-process! (get-in ast [:params :payload]))})
Not Working:
(pc/defresolver process-batch [{:keys [ast]} _params]
  {::pc/input #{}
   ::pc/outuput [:batch-jobs/process-batch]}
  (let [payload (get-in ast [:params :payload])]
    {:batch-jobs/process-batch (auth/batch-process! payload)}))
#2022-09-1415:13Bjƶrn EbbinghausYou have a typo: ::pc/outuput#2022-09-1415:14Bjƶrn EbbinghausIn the first case pathom can infer the attributes you produce by itiself, because you return a map directly.#2022-09-1415:27njjI fixed the typo šŸ˜„, I had it correct in the local code#2022-09-1415:28njjso for the 2nd one (not working), should I be more explicit about the map that this endpoint returns?#2022-09-1419:16wilkerlucionot sure why it wouldn't work, the code looks equivalent to me, can you send a complete repro? maybe the problem is somewhere else#2022-09-1419:17wilkerlucio(and I see you calling it batch, but I don't see the ::pc/batch? true flag, wondering if you are trying to use batch somehow, or its just a name coincidence)#2022-09-1519:25njjthats just a name coincidence#2022-09-1710:22roklenarcicI have tons of namespaces that have resolvers and each has a (def resolvers [….]) at the end, and there’s a central NS that has a parser that collects all these vars. The problem is that in dev workflow, when I edit and save namespaces that have resolvers, the parser NS that uses them doesn’t get reloaded, so I don’t get the changes until I manually reload the parser namespace. What strategy are you using to tackle this problem?#2022-09-1710:37sheluchinhttps://clojurians.slack.com/archives/C68M60S4F/p1661965659644209#2022-09-1711:20roklenarcicOk so this basically boils down to a macro that offloads code from defmutation body into a separate defn so when namespace gets reloaded, the new defn gets called#2022-09-1712:25Mark WardleAn alternative approach that works for me is integrant. In general, my resolvers are usually very thin wrappers around a range of different services, so in the end, I often need to both restart those services AND reload the lists of resolvers. As such, I simply use integrant to stop and start everything, and it takes only a few moments, and I know everything is registered properly. e.g. see my integrant configuration: the potentially dynamic list of resolvers https://github.com/wardle/pc4/blob/738fbb9c9d42ec7c3af78aab45ef5267a84fe63f/pc4-server/resources/config.edn#L112 the pathom environment: https://github.com/wardle/pc4/blob/738fbb9c9d42ec7c3af78aab45ef5267a84fe63f/pc4-server/resources/config.edn#L122 the aero reader for referencing integrant components https://github.com/wardle/pc4/blob/738fbb9c9d42ec7c3af78aab45ef5267a84fe63f/pc4-server/src/com/eldrix/pc4/system.clj#L139 [for components that generate resolvers dynamically] and the aero reader for referencing resolvers listed in clojure namespaces: https://github.com/wardle/pc4/blob/738fbb9c9d42ec7c3af78aab45ef5267a84fe63f/pc4-server/src/com/eldrix/pc4/system.clj#L142 so I can be explicit in the list of resolvers to register.#2022-09-1712:30Mark WardleI'm sure you could do the same with some other component library. And you can usually map a hotkey to do the 'stop','load namespace','start' to make it very quick and easy.#2022-09-1916:10Ben GrabowI normally use a 0-arity defn instead of a def to hold the collections of resolvers. Then when I create a Pathom index I call each defn and pass the result to pci/register. I'm not sure if this becomes performance intensive to rebuild the index every time a resolver changes, but in my experience with ~10s of resolvers its basically instant.#2022-09-2117:35pithylessI pass in a 0-arity function that returns the actual pathom-env to any pathom interface functions. (ie. (boundary-interface (parser-env-fn)) instead of (boundary-interface parser-env)) If all the resolver collections are written as functions (vs cached defs), then you will always have up-to-date parsers. The only issue is performance, since you do not want the indexes to be parsed and rebuilt with every request. I solve this problem with a configuration flag:
(if (:reload-on-request? cfg)
  (fn [] (new-parser-env-fn))
  (constantly (new-parser-env-fn)))
The trick is constantly will cache the results of calling the function once, while the dev mode will rebuild the index with every request.
#2022-09-2117:36roklenarcicthat’s very nice#2022-09-1902:45Joe R. SmithThis is probably too ambiguous a question, but, I'm using the Datomic dynamic resolver and pbip/mutation-resolve-params and it will resolve nested ::pco/params for a mutation, but not for a resolver. I.e., if the top-level key in the params doesn't return one of the nested attributes, the Datomic dynamic resolver finds them, but only for mutations.e#2022-09-1903:02Joe R. Smithunrelated, what is the "datomic-source" arg supposed to be? https://github.com/wilkerlucio/pathom3-datomic/blob/main/src/main/com/wsscode/pathom3/connect/datomic.clj#L193#2022-09-1904:30Joe R. Smithpathom3-datomic's compute-dynamic-nested-requirements doesn't work with optional input (e.g., (pco/? :some/thing)#2022-09-1904:49Joe R. Smithas a result I've switched to using a mutation for compiling "reports" and resolved params#2022-09-2014:36jacekschaeAre there any helpers when it coms to converting ns keys in maps before putting them in a graph? Assuming following result from a DB
[{:account/verified true,
  :account/full-name "First Second",
  :account/account-id #uuid"01835ba0-07c0-835d-bd8e-e476b19104598"}]
Is there any recommended way to expose this as:
[{:my-app.account/verified true,
  :my-app.account/full-name "First Second",
  :my-app.account/account-id #uuid"01835ba0-07c0-835d-bd8e-e476b19104598"}]
#2022-09-2014:45sheluchinYou can use https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/update-keys I have something like this:
(defn update-map-kw-ns                           
  "Set the namespace of every key in the map."   
  ([m]                                           
   (update-keys m #(keyword (name %))))          
  ([m ns-str]                                    
   (update-keys m #(keyword ns-str (name %)))))) 
#2022-09-2106:34jacekschaeThanks, yeah i have something similar with reduce-kv yet was wondering if there is anything baked-in in Pathom.#2022-09-2119:55lilactownthis is fascinating: https://www.sqlite.org/np1queryprob.html#2022-09-2218:38souenzzoas good as datomic (on-prem)#2022-09-2218:41souenzzocc @U9ABG0ERZ#2022-09-2620:00wilkerlucioadd this to the docs at https://pathom3.wsscode.com/docs/resolvers/#dont-overuse-batch#2022-09-2119:56lilactown> Using the N+1 Query pattern in Fossil does not harm the application. But the N+1 Query pattern does have benefits. For one, the section of the code that creates the timeline query can be completely separate from the section that prepares each timeline entry for display. This provides a separation of responsibility that helps keep the code simple and easy to maintain. Secondly, the information needed for display, and the queries needed to extract that information, vary according to what type of objects to be shown. Check-ins need one set of queries. Tickets need another set of queries. Wiki pages need a different query. And so forth. By implementing these queries on-demand and in the part of the code dealing with the various entities, there is further separation of responsibility and simplification of the overall code base. > So, SQLite is able to do one or two large and complex queries, or it can do many smaller and simpler queries. Both are efficient. An application can use either or both techniques, depending on what works best for the situation at hand.#2022-09-2119:57lilactownI'm posting this here because this is a common problem in GraphQL and I imagine in pathom too, which it sounds like if you can store all your data in sqlite, is not a problem at all šŸ˜„#2022-09-2119:59wilkerlucioyup, really fascinating, although pathom has features to deal with batching, not using it is simpler, so this is a great target to pathom processing without much optimization fuzz :)#2022-09-2211:40Mark WardleIn relation to batching, I have a large number of encounters, each of which have to-many relationships to a number of 'forms' containing data items. So, I am dealing with the n+1 problem at the moment. My issue is that I'm not aware that my SQL "IN" can return the results in the correct order. Is there a clever way of using SQL to return ordered results for an "IN", or do I group the results and then re-order based on the original order using clojure - I'm thinking to group, and then a reduce across the original sequence of identifiers in order to lookup the results?#2022-09-2212:34wilkerluciothere is a helper called restore-order that can help, check: https://pathom3.wsscode.com/docs/resolvers/#matching-results#2022-09-2213:38Mark WardleThanks Wilker! #2022-09-2611:05Ben SlessFor some use case I have in mind, I wonder if Pathom can be the right solution - I have data I get from a server as a feed and ingest it in a datascript DB. For getting older data and some other queries, I need to query the server via HTTP. The results can then be cached in the local datascript DB. It sounds like resolvers map on to the problem, where data is resolved from the local DB or remote server if it isn't found. It's similar to the pagination example in the documentation, where a resolver would have to return the query results in a map of current/more?#2022-09-2615:39lilactownit sounds like the primary problem is resolving the data either from cache, or network if it's not found. is that right?#2022-09-2615:43lilactownare you using the datascript DB to do something like render a UI, or is it a cache of some other service?#2022-09-2616:25Ben SlessI want to use datascript as a local cache to render a UI, and if data does not exist fetch it from network, yes#2022-09-2621:14lilactownthe problem then is that you'll want to access the cache synchronously, while the data fetching will be async.#2022-09-2621:15lilactownpathom won't help you much here because it will return a (async) promise no matter#2022-09-2621:16lilactowntypically what people do is they subscribe their UI to the cache, and then in the background they fetch and populate the cache. So initially the UI will render with no results, and then later will re-render once the cache is populated with the response from the server#2022-09-2621:17lilactownit's typical to do this fetching on some action, like on page navigation, button click, app startup, etc. so that you're not refetching the data all the time, but also you give the user the ability to refetch#2022-09-2808:25Ben SlessThis sounds right, although I can trigger the app state updates asynchronously too. I'll have to experiment and see what produces saner code Thanks!#2022-09-2803:45nivekuilI'm trying out full-stack dynamic resolvers in fulcro, but it looks like it will slow down first load because the app initialization needs to be async to wait one round trip for the indexes to be fetched first. Any better way to do it? ideally the indexes could be fetched along with the html/css/js and held in a CDN#2022-09-2814:42wilkerluciowhat do you mean by "full-stack dynamic resolvers"?#2022-09-2822:58nivekuiluse boundary-interface as the remote instead of the normal http remote#2022-09-2823:00wilkerluciogotcha, so, it really depends on when/how you initialize it, I dont have any mechanism to cache, but there is a way to ā€œtrick itā€, by wrapping the boundary interface with a fn, and intercepting the request for indexes#2022-09-2823:01wilkerlucioIm afk now, but I can send an example later#2022-09-2823:02nivekuilif you have an example for getting this working at all I'd love to see it. my attempt failed, I believe I couldn't figure out why the boundary-interface seemed to work in the repl but when fulcro calls it a lot of attributes return "Batch error: cannot write Function"#2022-09-2823:03nivekuilalso there was ::p/not-found coming from somewhere and I don't even see that in the pathom code#2022-09-2823:04wilkerluciothe error sounds like some encoding issue, which could explain the fail on remote, but not in the repl#2022-09-2823:05wilkerluciop not found is weird, thats not a thing in pathom 3#2022-09-2823:06nivekuilwell, when I say repl I mean cljs browser repl, and it's using the same transit stuff as the normal fulcro http remote#2022-09-2823:08nivekuil#2022-09-2823:08nivekuilproof of not-found#2022-09-2823:09nivekuilthat's in fulcro inspect#2022-09-2823:09wilkerlucioare you using pathom3 in both ends?#2022-09-2823:09nivekuilyes#2022-09-2823:09wilkerluciobecause thats a regular pathom na#2022-09-2823:09nivekuiloh wait it's fulcro inspect#2022-09-2823:09wilkerlucioms#2022-09-2823:09wilkerlucions#2022-09-2823:09nivekuilthat must be using pathom2#2022-09-2823:10wilkerluciomakes, could be#2022-09-2823:14nivekuilso if the error-stack only has .js files then the error happened on the client? does it also do full-stack stacktraces?#2022-09-2823:27nivekuilfull code
(defn api-request [body]
  (-> (js/fetch (str js/window.location.origin "/api")
                #js {:method  "POST"
                     :headers #js {"Content-Type" "application/transit+json"
                                   "Accept"       "application/transit+json"}

                     :body (fulcro-transit/transit-clj->str body)
                     })
      (p/then (fn [response] (.text response)))
      (p/then' (fn [response] (log/spy (fulcro-transit/transit-str->clj response))))))

(def pathom
  (p/let [api (pcf/foreign-register api-request)]
    (let [env (-> #_(p.plugin/register [init-state-plugin])
                  (pci/register [api])
                  )]
      (p.a.eql/boundary-interface env))))
#2022-09-2900:26nivekuilactually strike all that, I realized that I actually had the resolver definition wrong but this was asymptomatic unless using dynamic resolvers#2022-09-2900:27nivekuilstill, doesn't work with fulcro since the cljs writer tries to serialize a Function for some reason#2022-09-2900:27wilkerlucioIm very interested in your experience, afaik no one is using it that way, so I expected rough edges, and interested in fixing them :)#2022-09-2900:27wilkerluciothat rel with fulcro is something we can investigate#2022-09-2900:28wilkerluciodoes your transit encoder understands pathom resolvers/mutations?#2022-09-2900:28wilkerluciothey are a special type, and pathom providers readers/writters for them#2022-09-2900:29nivekuilyeah, I have pcot/write-handlers merged in#2022-09-2900:30wilkerluciocan you provide an example repo with the things you are trying? I like to see closer#2022-09-2900:30wilkerlucioand try myself#2022-09-2900:36nivekuilit would have to be the whole fulcro project, I can try making a smaller example later#2022-09-2900:53nivekuilI see the problem, it's in the AST that is passed by the boundary interface to the http request. looking at the ast, :component is actually a function, I think it should be a symbol?
{:type :join,
                                     :dispatch-key :user/views,
                                     :key :user/views,
                                     :query [:view/id :view/title],
                                     :component app.ui.view-meta/UserView,
                                     :children [{:type :prop, :dispatch-key :view/id, :key :view/id}
                                                {:type :prop,
                                                 :dispatch-key :view/title,
                                                 :key :view/title}]}
#2022-09-2900:55nivekuilI guess that ast needs to be cleaned up before it gets passed to transit#2022-09-2901:18nivekuilso I call (boundary-interface query) and this is correct, then pathom calls the foreign-registered api-request with a bad AST#2022-09-2901:18nivekuilI think I can instead of using EQL, pass a cleaned up AST directly to the boundary interface#2022-09-2901:19wilkerlucioyes you can :)#2022-09-2901:20nivekuilso I'm guessing that the EQL->AST writes the component metadata that fulcro attaches as functions and that's what's breaking transit#2022-09-2901:21wilkerlucioquite possible#2022-09-2901:21nivekuilalso the same for the AST that is passed to the remote#2022-09-2901:21nivekuilnot sure what the best way to clean it up is though, spectre?#2022-09-2901:22wilkerlucioone good thing is to make transit more resilient, by setting up a default handlwe#2022-09-2901:22wilkerlucioon the writer#2022-09-2901:28nivekuiloh, I remember you mentioning this now.. the default-handler doesn't work in cljs right#2022-09-2901:34nivekuilah, ok it just works with transito šŸ™‚#2022-09-2901:36nivekuilhttps://gist.github.com/nivekuil/579107f0c0e1274cb0c4e9785737f0a9#2022-09-2901:52nivekuilthis look like a bug?
(p/let [res1 (api-request [{:app.api.wikidata/recent-tags [{:>/tag {:tag/qid [:wiki/label]}}]}])
          res2 (api-request [{:app.api.wikidata/recent-tags [{:tag/qid [:wiki/label]}]}])]
    (println res1)
    (println res2))
{:app.api.wikidata/recent-tags [{:>/tag {}} {:>/tag {}} {:>/tag {}} {:>/tag {}} {:>/tag {}} {:>/tag {}}]}

{:app.api.wikidata/recent-tags [{:tag/qid {:wiki/label flag semaphore}} {:tag/qid {:wiki/label microscope}} {:tag/qid {:wiki/label WebTransport}} {:tag/qid {:wiki/label benchmark}} {:tag/qid {:wiki/label Craigslist}} {:tag/qid {:wiki/label Z shell}}]}
#2022-09-2901:52nivekuiladding the placeholder should not affect results?#2022-09-2901:53wilkerlucioI see a join missing a vector after :>/tag#2022-09-2901:55nivekuilah yeah. oops#2022-09-2901:55nivekuilbrain fried#2022-09-2902:16nivekuilok this one, 50% sure I didn't mess up
(p/let [p pathom
          res1 (p [{[:wiki/id 9061] [{:wiki/claims [:wiki/predicate {:>/wiki-item [:wiki/id :wiki/label]}]}]}])
          res2 (p [{[:wiki/id 9061] [{:wiki/claims [:wiki/predicate :wiki/id :wiki/label]}]}])]
    (println res1)
    (println res2)
    )
{[:wiki/id 9061] {:wiki/claims [{:wiki/predicate 106} {:wiki/predicate 737} {:wiki/predicate 106} {:wiki/predicate 106} {:wiki/predicate 101} {:wiki/predicate 106} {:wiki/predicate 31} {:wiki/predicate 800} {:wiki/predicate 106} {:wiki/predicate 101} {:wiki/predicate 106} {:wiki/predicate 106} {:wiki/predicate 737} {:wiki/predicate 106} {:wiki/predicate 106} {:wiki/predicate 101} {:wiki/predicate 800} {:wiki/predicate 800} {:wiki/predicate 737} {:wiki/predicate 101} {:wiki/predicate 800} {:wiki/predicate 737} {:wiki/predicate 361}]}}

{[:wiki/id 9061] {:wiki/claims [{:wiki/predicate 106, :wiki/id 188094, :wiki/label economist} {:wiki/predicate 737, :wiki/id 76725, :wiki/label Max Stirner} {:wiki/predicate 106, :wiki/id 201788, :wiki/label historian} {:wiki/predicate 106, :wiki/id 4964182, :wiki/label philosopher} {:wiki/predicate 101, :wiki/id 159810, :wiki/label economy} {:wiki/predicate 106, :wiki/id 2306091, :wiki/label sociologist} {:wiki/predicate 31, :wiki/id 5, :wiki/label human} {:wiki/predicate 800, :wiki/id 58784, :wiki/label Das Kapital} {:wiki/predicate 106, :wiki/id 3242115, :wiki/label revolutionary} {:wiki/predicate 101, :wiki/id 5891, :wiki/label philosophy} {:wiki/predicate 106, :wiki/id 1930187, :wiki/label journalist} {:wiki/predicate 106, :wiki/id 36180, :wiki/label writer} {:wiki/predicate 737, :wiki/id 9235, :wiki/label Georg Wilhelm Friedrich Hegel} {:wiki/predicate 106, :wiki/id 49757, :wiki/label poet} {:wiki/predicate 106, :wiki/id 82955, :wiki/label politician} {:wiki/predicate 101, :wiki/id 8134, :wiki/label economics} {:wiki/predicate 800, :wiki/id 40591, :wiki/label The Communist Manifesto} {:wiki/predicate 800, :wiki/id 295347, :wiki/label Economic and Philosophic Manuscripts of 1844} {:wiki/predicate 737, :wiki/id 76422, :wiki/label Ludwig Feuerbach} {:wiki/predicate 101, :wiki/id 21201, :wiki/label sociology} {:wiki/predicate 800, :wiki/id 470600, :wiki/label The German Ideology} {:wiki/predicate 737, :wiki/id 1170769, :wiki/label The Essence of Christianity} {:wiki/predicate 361, :wiki/id 1518091, :wiki/label Marx siblings}]}}
#2022-09-2902:18wilkerluciocheck after :>/wiki-predicate, I think its missing the vector there#2022-09-2902:19nivekuil:>/wiki-item is the placeholder#2022-09-2902:19nivekuilit's in the same level as :wiki/predicate#2022-09-2902:19wilkerlucioosrry, I mistyped, haa, but check after :wiki/predicate#2022-09-2902:20nivekuilyeah that should be correct syntax? [:user/id {:user/city [:city/name]}]#2022-09-2902:21wilkerlucioah, sorry, I misread, reading from the phone,rsrs#2022-09-2902:21nivekuilI wonder if it's not treating it as a placeholder but as a real join#2022-09-2902:22wilkerlucioir might be a bug#2022-09-2902:26nivekuilthe placeholder query works correctly through pathom-viz.. doesn't that use the same foreign env/boundary resolver#2022-09-2902:27wilkerlucioit may be something in the foreign integration#2022-09-2902:28wilkerlucioyou are using pathom viz connected to the cljs env or to the server env?#2022-09-2902:28nivekuilserver env#2022-09-2902:28nivekuilI didn't know you can connect it to the cljs env actually#2022-09-2902:38wilkerlucioyou can do with the pathom viz connector#2022-09-2902:39nivekuilah right, that has the ip/host issues we went over earlier.. I should get around to a PR sometime#2022-10-0313:52mitchelkuijpers@U066U8JQJ I also ran into some issues with a placeholder something seems to go wrong with the foreign interface and placeholders. It is fixed if I query the properties one place higher then the placeholder#2022-10-0313:54mitchelkuijpersso
{:foo/id {:>/placeholder [:foo/name]}}
Fails with cannot find path to [:>placeholder :foo/name]
{:foo/id :foo/name {:>/placeholder [:foo/name]}}
Fixes it
#2022-10-0316:07wilkerluciook, that totally looks like a bug#2022-10-0316:08wilkerluciocan you please start an issue about it?#2022-10-0409:11mitchelkuijpersYes I managed to reproduce it in a small reproducible case: https://github.com/wilkerlucio/pathom3/issues/159#2022-10-0411:57wilkerluciothank you!#2022-09-2806:40nivekuilseems like I can get some requests working but merging into fulcro is broken.. anyone try this yet?#2022-09-2818:49Bjƶrn EbbinghausYou can look at the server-side render stuff from Fulcro. There are functions to encode and decode initial-state into the HTML. So you have some state without having to make a request. I use it for translation strings for example. #2022-09-2822:58nivekuilyeah maybe SSR does something special#2022-09-2923:01nivekuilis there a function to convert data->eql out there?#2022-09-2923:02wilkerlucioyes: https://cljdoc.org/d/com.wsscode/pathom3/2022.08.29-alpha/api/com.wsscode.pathom3.format.eql#data-%3Equery#2022-09-2923:02nivekuilnice, glad I asked :) was looking in the eql lib#2022-09-3018:43souenzzoI think that this function can be added to https://github.com/edn-query-language/eql The pull question should be simple šŸ™‚ https://github.com/lilactown/pyramid/commit/8fb0b8c766094618c5a67e70bd72dbca470553de#2022-10-0201:26nivekuilthat function also looks handy, someone who actually wrote it should put up a PR :)#2022-10-0608:10Eric DvorsakIs there a way in pathom to batch n+2 attributes? Eg I am exposing a db that has a table for an entity, and multiple tables for "subtypes" of that entity. I have a resolver that outputs {:entity-subtypeA [:subtypeA/id]} with :entity/id and :entity/subtype as inputs. So far so good pathom resolves all the attributes of subtypeA that I query. The problem is that when querying a list of entities, the resolver for subtypes is batched, but the subsequent calls for the attributes are not and one call is made for each entity#2022-10-0612:00wilkerlucioin Pathom 3, it should batch also in the next level, as long as both resolvers support batching, it would be a two step batching, the first batch would get the items of the list, and the second a single batch for every sub-children#2022-10-0612:02Eric Dvorsaksub children should be in a different batch depending on their type i'll see how pathom3 does it once I migrated, almost done with the pathom3 fulcro-rad-demo branch#2022-10-0613:47wilkerluciothat's fine to be different batches, Pathom 3 does batching in a way that's very differnet from what Pathom 2 did, the Pathom 3 way is much more flexible#2022-10-0613:48Eric Dvorsaknice! can't wait to see it working#2022-10-0613:48Eric Dvorsakcurrently I'm struggling to get the db connection in the env with fulcro new-processor , seems like the example in fulcro-rad-demo doesn't work#2022-10-0615:33Eric Dvorsakoh it's because I was trying the query in pathom viz and it does have the full context#2022-10-0616:06Eric Dvorsaknice it is batching now#2022-10-0618:21Eric DvorsakI'm trying to figure out how to get pathom viz working, seems like it doesn't get the db in the env when using the new-processor https://github.com/fulcrologic/fulcro-rad-demo/compare/develop...pathom3#diff-d08b0b392ee3f4c4541e3539b449936f8c6835533f211df066764c5cda4e045aR27 I added the line (p.connector/connect-env {::pvc/parser-id `env}) directly here https://github.com/fulcrologic/fulcro-rad/blob/develop/src/main/com/fulcrologic/rad/pathom3.clj#L110#2022-10-0621:17wilkerluciogiven its a fulcro setup thing I think its better to ask about that on #C68M60S4F#2022-10-0618:48Eric DvorsakIs the docstring here up to date here? https://github.com/wilkerlucio/pathom-viz-connector/blob/master/src/com/wsscode/pathom/viz/ws_connector/pathom3.cljc#L81 It looks like there's no parser in pathom3?#2022-10-0620:54wilkerluciothat's outdated, sorry about that#2022-10-0620:54wilkerlucioI guess I copied over and oversight that#2022-10-0620:54wilkerlucioI can fix in a bit#2022-10-0709:08Eric Dvorsaknice#2022-10-0717:30Eric DvorsakIt looks like when something goes wrong with a query on pathom viz there is no way to be able to run a query again without restarting the app? in the console I see
Error handling response
{:com.wsscode.async.processing/request-id
 #uuid "94df6680-eff5-41d4-aff9-7b99755da920",
 :request-keys
 (:com.wsscode.node-ws-server/client-id
  :com.wsscode.async.processing/request-id
  :edn-query-language.core/query
  :com.wsscode.pathom.viz.ws-connector.core/type), 
 :timeout 5000}
being printing regularly, and the Run query button stays greyed out indefinitely, even when doing a force reload. Am I missing something?
#2022-10-0717:31Eric Dvorsak(after waiting a bit more and anoter force reload Run query was available again)#2022-10-1420:07caleb.macdonaldblackWould love to get peoples thoughts on this toy project I put together that uses Pathom3, Reagent and Datascript to create a SPA. https://www.reddit.com/r/Clojure/comments/y43tw6/pathom3_datascript_reagent_spa_toy_project/#2022-10-1420:07caleb.macdonaldblackhttps://github.com/CalebMacdonaldBlack/reav-example#2022-10-1516:41caleb.macdonaldblack@wilkerlucio if you haven’t already, you should consider applying for https://www.clojuriststogether.org/news/clojurists-together-long-term-funding-for-2022-2023/#2022-10-1520:01caleb.macdonaldblackIs there anything concerning about calling p.eql/process within a mutation or resolver by passing the env param?#2022-10-1820:02wilkerlucioits a supported feature, it should be fine to call p.eql/process from inside#2022-10-1520:03caleb.macdonaldblackBasically I’m trying to abstract some mutations. A mutation for creating a child entity and a mutation for create a parent entity. The create-parent mutation can also create its children by processing a create-child mutation. I know I can call the mutation as a function, however I’m resolving params into my mutations.#2022-10-1709:34Eric DvorsakI have a query that the planner resolve correctly but inneficiently. Instead of using an id-resolver + alias it starts with a longer path which results in a few queries instead of one one the db. Is there a way to understand why the planner decided to take the long path to resolve an attribute and to force it to take the short path?#2022-10-1710:02sheluchinhttps://pathom3.wsscode.com/docs/resolvers/#priority-number#2022-10-1710:54Eric Dvorsakthanks, didn't know about priorities, the issue here though is that both chains are starting with the same resolver#2022-10-1900:31wilkerluciohello Eric, yes, you right, without the weight tracker, Pathom by default will only try to prioritize via explicit declarations, but I see the point in your case, when some options start the same, kinda makes sense that the smaller should go first#2022-10-1900:32wilkerluciothat said, are you using latest pathom version? and have you tried turning on the experimental path optimizations? I think in your case there, if you turn them on, you might not have these duplicated subpaths#2022-10-1900:32wilkerlucioplease let me know if you have a chance to try it, wonder if will affect your planning tree#2022-10-1900:33wilkerlucio(to enable experimental optimizations, add to your env: :com.wsscode.pathom3.connect.planner/experimental-branch-optimizations true)#2022-10-1907:58Eric Dvorsak@U066U8JQJ this is with both optim and weight plugin (graph and chosen path are the same without optim)#2022-10-1907:59Eric Dvorsakoptim alone is taking a longer path (and the graph is exactly the same as no optim/no weight plugin)#2022-10-1713:36mbjarlandso I ran into this s.o. question and feel quite inadequate with my answer. Figured somebody here would know what ::provides actually does and could provide a better answer .#2022-10-1714:50Eric Dvorsaklooking at the source provides is the output converted by a function called query->shape-descriptor which Convert pathom output format into shape descriptor format#2022-10-1714:51Eric Dvorsak`Shape descriptor is a format to describe data. This format optimizes for fast detection of value present given a shape and a value path.`#2022-10-1715:01mbjarlandOk - guess I feel somewhat better then. That's not too far from the comment I made on that s.o. question. Thank you for the check!#2022-10-1900:29wilkerlucioyeah, all good, just add my entry there, hope it can still helps šŸ™‚#2022-10-1901:42wilkerlucio#2022-10-1909:13roklenarcicHow do you write a resolver for a recursive datastructure?#2022-10-1911:07donavanDoes this help? https://edn-query-language.org/eql/1.0.0/specification.html#_recursive_queries#2022-10-1911:07donavanIt’s relatively straightforward#2022-10-1911:33dvingohttps://pathom3.wsscode.com/docs/eql/#recursive-queries#2022-10-1912:02roklenarcicWell how do I tell Pathom that the output of the resolver is a nested tree?#2022-10-1912:03roklenarcicSo obviously you can make a resolver that has the output [:id {:children [:id]}]#2022-10-1912:04roklenarcicthat resolver can answer a recursive query…#2022-10-1912:04roklenarcicBut it will be called for each node once. And I get the whole tree in a single DB query.#2022-10-1914:35dvingoif you return the entire tree pathom will just return it and not continue calling the resolver#2022-10-1914:36dvingoi've used that in the past as an optimization - calling db/pull directly instead of having nested pathom calls#2022-11-0415:52Eric DvorsakIf you want to keep the decomposition into resolvers, you can also leverage batching in pathom3. let's say you have a resolver that gets a list of IDs, one that gets the attributes, and maybe a nested one that gets nested attributes from the db It will only take 3 queries, 1 per resolver, if you write batch queries In pathom2 that wasn't working for nested attributes#2022-10-2000:46wilkerlucio#2022-10-2114:11caleb.macdonaldblackWas looking forward to this one. Nice work!#2022-10-2114:49wilkerlucioshout out to @U06B8J0AJ that brought the idea, also pointed the bug in the first release#2022-10-2211:57nivekuilthis is a really nice feature!#2022-10-2211:56nivekuilI'm trying the foreign dynamic resolver out again with the latest fixes, but I think there's a new problem: the new attribute-error-resolver is getting registered multiple times#2022-10-2211:57nivekuilugly stacktrace:
{:com.wsscode.pathom3.error/error-message "Assert failed: Tried to register duplicated resolver: com.wsscode.pathom3.connect.runner/attribute-errors\n(nil? (com.wsscode.pathom3.connect.indexes/resolver indexes op-name))", :com.wsscode.pathom3.error/error-data nil, :com.wsscode.pathom3.error/error-stack "
#2022-10-2211:58nivekuillooking at https://github.com/wilkerlucio/pathom3/blob/ad1a1cf71990c0ea4d763ae4cfc779f8be2a5b86/src/main/com/wsscode/pathom3/connect/runner.cljc#L1117 I guess the problem is I'm setting lenient on both the client and server env?#2022-10-2417:54wilkerluciohumm, makes sense, can you please open an issue? I have an idea what that might be about#2022-10-2907:34bennyI'm wrapping an API that returns IDs that could either be one "thing" or another. Is there a best practices how to work around that sloppy data structure?
{:name "foo", :contents [{:id 1, :type "person"} {:id 1, :type "group"}]}
I tried to add a content-to-person and content-to-group resolver that returns {:id 1, :type "person"} to {:person/id 1} and {:group/id 1} depending on the :type. But it's not guaranteed to contain a person nor a group so the eql-queries are rightfully barked at, when the data gets returned and a map doesn't contain a mix of both "things." Or would one just add {:person/id 1, :group/id nil} depending on the :type and just return nil for all additional attributes that can't be resolved? I'm lacking a "Maybe" type to query on which would infect my internal data structure. 😐
#2022-10-2908:30bennyFYI: For now to not get stuck I've opted to use an enveloping type that gets me my relevant attributes. It doesn't feel clean, but for now it gets the job done.#2022-10-2910:40roklenarcicunion query?#2022-10-2917:41bennyThanks. Seem like the right thing, except they don't work for me. I'll keep them in mind.#2022-10-3020:55caleb.macdonaldblack@wilkerlucio Inferring input from the destructuring doesn’t also infer nesting. What are your thoughts on a change to support this?
(pco/defresolver foo
  [{{{son :son} :father} :grandpa}]
  {:grandpa son})
=> #'com.calebmacdonaldblack.foo/foo
foo
=>
#com.wsscode.pathom3.connect.operation.Resolver{:config #:com.wsscode.pathom3.connect.operation{:input [:grandpa],
                                                                                                :provides {:grandpa {}},
                                                                                                :output [:grandpa],
                                                                                                :inferred-input [:grandpa],
                                                                                                :op-name com.calebmacdonaldblack.foo/foo,
                                                                                                :requires {:grandpa {}}},
                                                :resolve #object[com.calebmacdonaldblack.foo$foo__21752
                                                                 0x6f873565
                                                                 "
#2022-11-1422:08wilkerluciohello, that's by design#2022-11-1422:08wilkerlucioyes, we could support it, but I think at that point is better to have it explicitly, otherwise its kinda painful to read#2022-10-3020:56caleb.macdonaldblackSome examples of what I mean
(deftest extract-destructure-map-as-input-test


  (testing "Simple flat params"
    (is (= [:foo :bar]
           (extract-destructure-map-as-input
             '{foo :foo
               bar :bar}))))

  (testing "Nested input"
    (is (= [{:foo [{:bar [:buzz :y :a :b :c]}
                   :x]}]
           (extract-destructure-map-as-input
             '{{{fizz :buzz y :y :keys [a b c]} :bar x :x} :foo}))))

  (testing "Nested input with vector destructuring"
    (is (= [:foo]
           (extract-destructure-map-as-input
             '{[foo1 foo2] :foo}))))

  (testing "Nested input with vector destructuring and more nesting"
    (is (= [{:foo [:bar]}]
           (extract-destructure-map-as-input
             '{[{bar :bar}] :foo}))))

  (testing "Simple flat params with symbol keys"
    (is (= [:foo :bar]
           (extract-destructure-map-as-input
             '{:keys [foo bar]}))))

  (testing "Simple flat params with keyword keys"
    (is (= [:foo :bar]
           (extract-destructure-map-as-input
             '{:keys [:foo :bar]}))))

  (testing "Simple flat params with namespaced symbol keys"
    (is (= [:ns/foo :ns/bar]
           (extract-destructure-map-as-input
             '{:keys [:ns/foo :ns/bar]})))))
#2022-11-1616:13caleb.macdonaldblack@UPWHQK562 #2022-10-3108:31pieterbreedI'm using the ::runner/wrap-resolver-error plugin with lenient mode so that I can have warning logs when attributes do fail:
(-> (pci/register {:com.wsscode.pathom3.error/lenient-mode? true}
                  registry)
    (p.plugin/register {::p.plugin/id 'err
                        :com.wsscode.pathom3.connect.runner/wrap-resolver-error
                        (fn [_] (fn [env node error]
                                  (log/warn {::pathom-error error
                                             ::pathom-env   env
                                             :msg
                                             (format "Error resolving attribute '%s': '%s'."
                                                     node
                                                     (ex-message error))})))}))
I'm noticing that on attributes that fail outright (ie with exceptions) this log statement gets triggered, but surprisingly attributes with {:com.wsscode.pathom3.error/cause :com.wsscode.pathom3.error/attribute-unreachable} does not go through this "resolver-error" fn. Is this intended behaviour? If so, why?
#2022-11-1422:10wilkerlucioyes, this is currently by design, but up for debate, what you think about opening a discussion (https://github.com/wilkerlucio/pathom3/discussions) so we can talk more about it?#2022-11-0219:21Eric Dvorsakis it possible to limit which resolvers can be used as entry points for a parser while still using all of them for the resolution of attributes? eg it's really complex to come up with an authorization system that would efficiently (both in terms of code and execution time) protect attributes some resolvers could be used directly in a query and require complex authorization code when in practice they are always used from a context where parent attributes have already been authorized#2022-11-0221:50roklenarcicthat would break the general idea of it all it seems#2022-11-0222:25souenzzohttps://github.com/souenzzo/eql-style-guide/issues/4#2022-11-0309:06Eric Dvorsak@U2J4FRT2T yes I think the wrapped parser from the readme could be an interesting approach. Just not sure how to make it performant eg. work well with batched queries. In my case the user have access to some top level entities. I will need to find a solution so that when the child entities are accessed within the context of a top level entity this is grabbed from the env. When a child is accessed directly the resolver should find the parent (sometimes a few levels of nesting)#2022-11-0315:24Pragyan TripathiI have gotten around it by creating environment dynamically based on the permissions the user has. We register the pathom environment as one of the middleware step. and then use that environment to resolve the queries. As environment doesn’t have resolvers that the user is not supposed to resolve… It fails with 404…#2022-11-0317:42caleb.macdonaldblack@U03K8V573EC My thoughts are that the index is a graph, not a tree. So there may not be a parent as it could be cyclic. :house/occupants -> occupant/house -> house/occupant. #2022-11-0317:46Eric Dvorsakwell in my model that would rather be: :house/occupants -> user/house (it looks like a many-to-one relationship) if we go deeper let's say you have house/rooms -> room/house and room/windows -> window/room only occupants of the house should be able to access these items only an occupant should be able to query {[:window/id x] [:window/attributeX]}#2022-11-0317:54Eric Dvorsakso I'm looking for a way to make this as automated as possible the approach I'm experimenting with is to leverage resolvers so the window/authorized? resolver will need room/authorized? resolver as input and the room/authorized? will need house/authorized? house/authorized? simply checks that the current user is an occupant this won't work with resolvers alone because if authorized? is an input of the id-resolvers, you can't resolve eg the window/room to resolve room/authorized? so what I'd try is to augment the query to add :authorized attribute to the queries and then clean up results when it's false, preferably at entity level rather than after full resolution alternatively something I already got working but didn't like was to have an unauthorized resolver for entities, which nests attributes in eg :window/unauthorized, then a resolver that takes that nested input + parent/authorized as input and returns the regular output attributes. this resolves the parents authorization recursively (you'd have to also forbid direct access to the unauthorized resolvers else someone can request window/unauthorized attributes to bypass)#2022-11-0323:11caleb.macdonaldblackI would solve this by creating an index based on permissions before processing the query. Find what top level resolvers the user has permissions for and then traverse the plan to figure out what can be accessed. Once you know what resolvers can be accessed, create a new index with just those resolvers and run the query #2022-11-0323:13caleb.macdonaldblackMaybe you can tag resolvers with some meta data or something to denote whether or not it’s a resolver that requires permissions. You would need to be careful because it seems like it would be easy to unintentionally make a connection from a resolver it has permission for, to one it doesn’t. #2022-11-0323:13caleb.macdonaldblackAnd your plan traversal would just assume it’s permitted because there is a relationship. #2022-11-0323:15caleb.macdonaldblackAnd if you want to use Pathom to to solve the problem of what resources your user has access to, then you can create a separate index for that for internal use only where the query is controlled. #2022-11-0323:16caleb.macdonaldblackSo one index determines the permissions and is only internally accessible. Then use the result of that to dynamically filter your other index of resolvers it can access. #2022-11-0323:18caleb.macdonaldblackThe main thing being the creation of the environment/index based on user permissions. Essentially what @U02JRAM6CBA is suggesting #2022-11-0410:16Eric DvorsakA separate index would not work, because the permissions are per entity not resolvers. So as the occupant of a house I should be able to query a room/window by indent IF it's in my house, but I shouldn't be able to query the neighbor's#2022-11-0410:19Eric Dvorsakwhat I like about my current poc is that it lets pathom deal with figuring out permissions. This means that i also benefits from caching and batching mechanisms (I'm using pathom3 which is much better at batching than pathom2)#2022-11-0413:18roklenarcicHow do you expect pathom planner to work around your system. Resolvers describe which attributes can be derived from other attributes, and you are attempting to twist that into something that gatekeeps access to a particular instance of information such as house/window 44 . It will not work. Ideas like hacking the index also don’t work. You’ll need to just check if entity is accessible in the resolvers. It’s just straightforward like that.#2022-11-0413:53Eric Dvorsak@U66G3SGP5 right now the only thing that doesn't work is that the runner uses the incomplete nested input, and I suspect it is related to https://github.com/wilkerlucio/pathom3/issues/167#2022-11-0414:03Eric Dvorsak> Resolvers describe which attributes can be derived from other attributes which is the reason I want to try to do it that way, because authorization in my use-case is derived from other attributes > you are attempting to twist that into something that gatekeeps access to a particular instance of information what I'm doing is resolving attributes in 2 steps, first step is a simple resolver that gets the attribute but is nesting them in an attribute/unauthorized key, second step is a resolver that takes the nested attributes as input and an authorization attribute and un-nests the attributes. the authorization attribute is resolved using the nested attributes, eg window/authorized? will take window/id and room/authorized? as input, room/authorized? will take room/id and house/authorized? house/authorized? will take house/id so let's say you get window/material for [window/id 1]:
regular resolver
{:window/unauthorized {:window/id 1 :window/material :wood :window/room 6}}
room resolver
{:window/unauthorized {:window/id 1 :window/material :wood :window/room 6 :room/unauthorized {:room/id 6 :room/house 2}}}
house-authorized (if not authorized, will throw and rest of query can't continue)
{:window/unauthorized {:window/id 1 :window/material :wood :window/room 6 :room/unauthorized {:room/id 6 :room/house 2 :house/authorized? true}}}
room authorized
{:window/unauthorized {:window/id 1 :window/material :wood :window/room 6  :room/id 6 :room/house 2  room/authorized? true :house/authorized? true}}}
window authorized
 {:window/id 1 :window/material :wood :window/room 6  :room/id 6 :room/house 2  room/authorized? true :house/authorized? true :window/authorized? true}}}
#2022-11-0414:04Eric Dvorsakofc you need to purge the query of attribute/unauthorized keys#2022-11-0414:41Eric Dvorsakactually I don't think my current issue is with indexed nested attributes but rather batches terminating too early#2022-11-1021:51caleb.macdonaldblack@U03K8V573EC how did you go with this auth stuff? #2022-11-1022:59Eric DvorsakIt works if you don't do batch resolvers#2022-11-1023:00Eric DvorsakI'm trying to find what is going wrong in batches, i had a workaround that was working 1 time ou of 3 šŸ˜…#2022-11-1023:01Eric DvorsakIt would be huge because it would be close to impossible to have that granularity of authorizations with grapqhl, especially while leveraging batches for performance at the same time#2022-11-1023:02Eric DvorsakOf course you can always denormalize things a bit#2022-11-0310:52Eric DvorsakIs it still possible in pathom3 to change the env in a resolver like in pathom2 by returning a modified env through the ::p/env key?#2022-11-0315:37henrikAFAIK, it’s not, but please correct me if I’m wrong @U066U8JQJ. For more self-contained stuff, you can ship in an atom in the env, which you can swap during processing.#2022-11-0317:28wilkerluciocorrect, not a feature in Pathom 3#2022-11-0417:04Eric Dvorsak@wilkerlucio I think I found an issue with batching in pathom3: I get a java.lang.IndexOutOfBoundsException from missing-maybe-in-pending-batch?#2022-11-0417:04Eric Dvorsakbatch minimal repro:
(pco/defresolver unauthorized-toy
  [env input]
  {::pco/input  [:toy/id]
   ::pco/batch? true
   ::pco/output [{:toy/unauthorized [:toy/name :toy/id :child/id]}]}
  [{:toy/unauthorized {:toy/name "Bobby" :toy/id 1 :child/id 1}}])

(pco/defresolver toy
  [env input]
  {::pco/input  [{:toy/unauthorized [:toy/name :toy/id :child/id :child/authorized?]}]
   ::pco/batch? true
   ::pco/output [:toy/name :toy/id :child/id]}
  (mapv :toy/unauthorized input))

(pco/defresolver child-toys
  [env input]
  {::pco/input  [:child/id]
   ::pco/batch? true
   ::pco/output [{:child/toys [:toy/id]}]}
  [{:child/toys [{:toy/id 1}]}])

(pco/defresolver unauthorized-child
  [env input]
  {::pco/input  [:child/id]
   ::pco/batch? true
   ::pco/output [{:child/unauthorized [:child/name :child/id :parent/id]}]}
  [{:child/unauthorized {:child/name "Bob" :child/id 1 :parent/id 1}}])

(pco/defresolver child
  [env input]
  {::pco/input  [{:child/unauthorized [:child/name :child/id :parent/id :parent/authorized?]}]
   ::pco/batch? true
   ::pco/output [:child/name :child/id :parent/id]}
  (mapv :child/unauthorized input))

(pco/defresolver auth-child
  [env input]
  {::pco/input  [:child/id :parent/authorized?]
   ::pco/batch? true
   ::pco/output [:child/authorized?]}
  #_(Thread/sleep 1000)
  [{:child/authorized? (:parent/authorized? input)}])

(pco/defresolver auth-parent
  [env input]
  {::pco/input  [:parent/id]
   ::pco/batch? true
   ::pco/output [:parent/authorized?]}
  (Thread/sleep 1000)
  [{:parent/authorized? true}])

(def env (-> (pci/register
              [unauthorized-toy
               toy

               child-toys
               unauthorized-child
               child
               auth-child
               auth-parent])
             (p.connector/connect-env {::pvc/parser-id `env-test})))
#2022-11-0417:05Eric DvorsakNon batch:
(pco/defresolver unauthorized-toy
  [env input]
  {::pco/input  [:toy/id]
   ::pco/output [{:toy/unauthorized [:toy/name :toy/id :child/id]}]}
  {:toy/unauthorized {:toy/name "Bobby" :toy/id 1 :child/id 1}})

(pco/defresolver toy
  [env input]
  {::pco/input  [{:toy/unauthorized [:toy/name :toy/id :child/id :child/authorized?]}]
   ::pco/output [:toy/name :toy/id :child/id]}
  (:toy/unauthorized input))

(pco/defresolver child-toys
  [env input]
  {::pco/input  [:child/id]
   ::pco/output [{:child/toys [:toy/id]}]}
  {:child/toys [{:toy/id 1}]})

(pco/defresolver unauthorized-child
  [env input]
  {::pco/input  [:child/id]
   ::pco/output [{:child/unauthorized [:child/name :child/id :parent/id]}]}
  {:child/unauthorized {:child/name "Bob" :child/id 1 :parent/id 1}})

(pco/defresolver child
  [env input]
  {::pco/input  [{:child/unauthorized [:child/name :child/id :parent/id :parent/authorized?]}]
   ::pco/output [:child/name :child/id :parent/id]}
  (:child/unauthorized input))

(pco/defresolver auth-child
  [env input]
  {::pco/input  [:child/id :parent/authorized?]
   ::pco/output [:child/authorized?]}
  #_(Thread/sleep 1000)
  {:child/authorized? (:parent/authorized? input)})

(pco/defresolver auth-parent
  [env input]
  {::pco/input  [:parent/id]
   ::pco/output [:parent/authorized?]}
  #_(Thread/sleep 1000)
  {:parent/authorized? true})

(def env (-> (pci/register
              [unauthorized-toy
               toy

               child-toys
               unauthorized-child
               child
               auth-child
               auth-parent])
             (p.connector/connect-env {::pvc/parser-id `env-test})))
#2022-11-0417:05Eric DvorsakQuery:
[{[:child/id 1] [:child/name {:child/toys [:toy/id :toy/name]}]}]
#2022-11-0417:09Eric DvorsakWhat I noticed is that when running batch resolvers, the toy resolver is ran in parallel with the unauthorized-toy resolvers so even though the planner found the right path with the right inputs for toy through unauthorized-toy, it tries to resolve it before it and end up with missing attributes. I suspect that the bug is somewhere in the planner rather than in missing-maybe-in-pending-batch? where the exception is caused by a path in batch-pending* that is shorter than the one in the env#2022-11-0418:36Eric DvorsakSeems like it might be a runner rather than planner issue since the snapshot plans in pathom viz are the same#2022-11-0418:42wilkerluciothanks, can you please open an issue for it?#2022-11-0418:55Eric Dvorsakyes https://github.com/wilkerlucio/pathom3/issues/169#2022-11-0418:55wilkerluciothanks!#2022-11-0419:14Eric DvorsakI thought that the planner was only running once in pathom3 but it's actually done on every subquery?#2022-11-0419:15wilkerlucioyeah, thats correct, but a bit more complicated than that#2022-11-0419:16wilkerluciopathom does verify (at parent planning) that a sub query is possible, so it fails early if it detects an impossible path at the subquery#2022-11-0419:16wilkerluciobut then later it plans again, because the plan depends on what data actually came (in contrast to what data it expected to have)#2022-11-0419:19Eric Dvorsakbut it's using the plan-cache* so not replanning from scratch right?#2022-11-0419:20wilkerluciocorrect, in case it matches the expectation it will possibly use from cache#2022-11-0419:22Eric Dvorsakdo you have an hunch on how this bug is happening? trying to get a grasp of the runner code to find it myself#2022-11-0419:24wilkerlucioafk at the moment, I hope I’ll be able to have a look by tomorrow#2022-11-0419:26Eric Dvorsakok I'll add my findings to the issue if I get something#2022-11-0512:20Eric Dvorsak@wilkerlucio so I think it might actually be the planner#2022-11-0512:36Eric DvorsakI haven't figured out yet how the planning works exactly but it definitely seems odd that one sub-plan is (2 unauthorized-toy) -> (1 toy) directly, while another is doing (unauthorized-child)->(child)->(auth-parent)->(auth-child) where (auth-child) result should have been the input of (1 toy)#2022-11-0512:37Eric Dvorsakit might just be working without batches if the planning goes depth first#2022-11-0708:23Eric Dvorsak@wilkerlucio I suspect the problem is that comput-run-graph is only generating a run plan for one level of the AST, so it misses nested inputs. It tries to resolve them on the next level but at this point the resolver that is missing these nested inputs is already planned in the previous level and decoupled from the subgraph that resolves them#2022-11-0708:45Eric DvorsakI suspect compute-nested-requirements might lead the planner to expect outputs without considering required inputs#2022-11-0713:35Eric Dvorsak@wilkerlucio I was able to successfully complete the query by passing the ::pcp/id-counter to the parser env directly and removing it from pcp/base-env as I suspected nodes were getting overwritten#2022-11-0713:44Eric Dvorsakalso I had to set pcp/optimize-graph? to false#2022-11-0713:48wilkerluciohello Eric, sorry, I wasn't able to sit at all at my computer this weekend, so I didn't had a chance to check, and glad you are making such investigation on it#2022-11-0713:49wilkerluciobut one thing, about optimize graph, I highly discourage disabling it, its a crucial part of the planner to stay performant, if its the case it works after removing, it may be a signal that some optimization might be broken, and if so, the best path is to figure what is broken there#2022-11-0713:50Eric Dvorsakyes I removed it to see if it was working without#2022-11-0713:51Eric Dvorsakbut I still think there is something wrong with the plan, I would expect a graph where toy/name is resolved by unauthorized/toy AND child/authorized?#2022-11-0713:52Eric Dvorsakwhat I haven't figured out yet is why the planner seems to find that path when planning toy/name, but doesn't add it to the plan and still comes up with just toy/unauthorized -> toy#2022-11-0717:05Eric Dvorsakmore or less back to zero. only certitude so far is that by tweaking missing-maybe-in-pending-batch? so that it doesn't try to subvec the path if pending path is shorter it sometimes solve the query (non-deterministic)#2022-11-0717:24Eric Dvorsak@wilkerlucio so my latest theory is that shape-reachable? is computing a whole new plan and determines that all the inputs of toy can be reached through unauthorized/toy (which is true if the planner was providing the plan for the nested ones) then the issue happens when the runner gets the result of unauthorized/toy and tries to run toy before a new round of planning that actually produces a plan that reaches the whole shape#2022-11-0812:04Eric Dvorsak@wilkerlucio been wrong so many times now I'm not sure what it's worth but I'm now suspecting the fact that :child/unauthorized is present twice in the graph (one for [:child/id 1] and one within :toy/unauthorized at some point during the graph run#2022-11-0420:35kendall.buchananWhat is the proper way to pass a param (e.g. pco/params) to an intermediate resolver?#2022-11-0514:07dvingoone technique is to collect all query params in all top level children of the root query and make them available in the environment, for example: https://github.com/holyjak/minimalist-fulcro-template/blob/7ca46fbbb53ac6d18ff54fd80f89b5109d502fe7/src/com/example/server/pathom.clj#L68#2022-11-0616:05kendall.buchananGreat idea, thank you.#2022-11-2913:54kendall.buchanan@wilkerlucio While I appreciated @U051V5LLP’s answer above, I’m a little surprised by the behavior. I’m finding, in practice, if I don’t have a query that immediately resolves to its query, the param is lost. In other words, the param is only good for one hop. Why is that the case?#2022-11-2913:59wilkerluciohello @U0HJA5ZQT, its a bit a tricky issue, originally most of the time params were used to things like pagination of a collection, where the param goes straight to the attribute that provides the collection, what I'm guessing in your case is that you may wanna use a param at some dependency of the attribute in which you place the param, is that a correct assessment?#2022-11-2914:01kendall.buchananIt’s simpler than that: I to provide the param at the beginning of an HTTP request, and know it’ll be available two, three resolvers in.#2022-11-2914:03kendall.buchananLemme mock it up real quick…#2022-11-2914:06kendall.buchanan
; Ident
{:user/username "
#2022-11-2914:07kendall.buchananAFAICT this doesn’t work. It only works if the ident leads directly to :checklists#2022-11-2914:10wilkerlucioI see, in this case you are in different entities, so by design this is not a way params are designed to flow, the params is an extra dimension for the attribute in which you use it, if you wanna flow it down its up to you, because if I try to do that at Pathom level, there will be ambiguities that I can't reconcile generically, because the user might not want those things down there, mixed with other (or even conflicting) params, makes sense?#2022-11-2914:12kendall.buchanani.e. I can choose to pass the param on?#2022-11-2914:17wilkerluciojust one thing, I was giving a bit more look at your example, and it does work in that case, but I think what you want is to have that down somehow, but I think the example may not represent what you trying to convey#2022-11-2914:17wilkerlucio
(ns com.wsscode.pathom3.demos.params-flow
  (:require
    [com.wsscode.pathom3.connect.indexes :as pci]
    [com.wsscode.pathom3.connect.operation :as pco]
    [com.wsscode.pathom3.interface.eql :as p.eql]))

(defn fetch-group-ids [username]
  )

(pco/defresolver resolver-1
  [_ {username :user/username}]
  {:group/ids (fetch-group-ids username)})

(defn fetch-checklists [group-ids my-param]
  (println "P" my-param))

(pco/defresolver resolver-2
  [env {group-ids :group/ids}]
  {::pco/output [:checklists]}
  (let [{:keys [my-param]} (pco/params env)]
    {:checklists (fetch-checklists group-ids my-param)}))

(def env
  (pci/register
    [resolver-1
     resolver-2]))

(comment
  (p.eql/process env
    {:user/username "
#2022-11-2914:17wilkerlucio
(p.eql/process env
    {:user/username "
#2022-11-2914:18wilkerlucioyou can use env to pass down, like we said before, I'm not sure if we can make it propagate down as a param, but it might be possible, I have to spend a bit more time to verify if forwarding params down is possible#2022-11-2914:19kendall.buchananMmmm. My real world example I’m testing with has resolvers four or five depths in.#2022-11-2914:19kendall.buchananI’ll have to look carefully for where the param drops off, and what’s different about that from the example I gave.#2022-11-2914:20kendall.buchananBut do you see any conceptual problems with the example above? I understand the notion of conflicting params… but is that not what namespacing is about?#2022-11-2914:20wilkerlucioin this example, we send params to :checklists, and read them at the resolver that provides :checklists, when there is this match, you should always have the param avaialble#2022-11-2914:21wilkerlucionot really, because that's a difference in context, its not about the param itself, but where its applicable#2022-11-2914:22wilkerlucioone thing that I still trying to figure is that if you want to forward it laterally (meaning its the same entity, but a different resolver in the execution graph) or if its vertical (flowing down to a different entity)#2022-11-2914:22wilkerlucioa more concrete example will be useful to understand your context#2022-11-2914:24kendall.buchananThank you for your help… lemme dig deeper, cause I’m not sure why the example I shared above worked, and what I’m working on does not. It’s a matter of tracing through more resolvers.#2022-11-2914:25kendall.buchananAh! Figured it out.#2022-11-2914:25kendall.buchananI had another resolver with the same output it was resolving to, and my tests were going in different directions.#2022-11-2914:26kendall.buchananBut yeah… the reason I originally kicked off this question WAS because of what you’re talking about: needing the param for an entity that did not match the provided query entity. šŸ‘ˆ#2022-11-2914:28wilkerlucioto give some reasoning on my design decisions, I like to keep things open so users have the flexibility to explore different ways, in Pathom case I think we still have so much to learn about how to model/query things, when regarding params, the same entity might want to use the same param with different values for different attributes, this means merging laterally would cause a conflict in such scenario#2022-11-2914:29wilkerluciosimilar idea goes when flowing down, how can I know that flowing is intentional? what if the user wants the param in a level ,but flowing down might configure something unintentionally? if we flow down, we do for every attribute? and how many levels it would flow down?#2022-11-2914:30kendall.buchananI suppose if the params had context around them…#2022-11-2914:31kendall.buchanan['(:checklists {:my-param true})] → There’s context around what that param was provided for.#2022-11-2914:31kendall.buchananI suppose if there was a way a resolver could access that somehow, knowing that the param was not intended for it.#2022-11-2914:32wilkerlucioone way is to make a plugin to add this context to env, maybe something like: {[PATH attribute] params}
#2022-11-2914:32wilkerlucioone other thing you can use is the query meta#2022-11-2914:32wilkerlucioits a way to add information for a entity section#2022-11-2914:33wilkerlucioso instead of [{(:checklists {:my-param true}) [:sub-query]}], you can try: [{:checklists ^{:my-param true} [:sub-query]}]#2022-11-2914:34wilkerluciothat's a different dimension, where you add data to a query fragment, instead of the attribute#2022-11-2914:34kendall.buchananAnd how is it accessed? (meta env)?#2022-11-2914:34wilkerluciolet me check that#2022-11-2914:38wilkerlucio
(ns com.wsscode.pathom3.demos.params-flow
  (:require
    [com.wsscode.pathom3.connect.indexes :as pci]
    [com.wsscode.pathom3.connect.operation :as pco]
    [com.wsscode.pathom3.interface.eql :as p.eql]))

(pco/defresolver resolver-1
  [env _]
  {:foo
   (let [{:keys [my-param]} (-> env
                                :com.wsscode.pathom3.connect.planner/graph
                                :com.wsscode.pathom3.connect.planner/source-ast
                                :query
                                meta)]
    (str "bar - " my-param))})

(pco/defresolver resolver-2 []
  {:checklists [{:id 1}]})

(def env
  (pci/register
    [resolver-1
     resolver-2]))

(comment
  (p.eql/process env
    ['{:checklists ^{:my-param true} [:foo]}]))
#2022-11-2914:39wilkerlucionote that, if you are sending this query over the wire, make sure its encoding/decoding meta, otherwise it wont work, an alternative is to send the AST instead of the query, which exposes the meta out (via boundary interface you can send {:pathom/ast AST})#2022-11-2914:40kendall.buchananOkay, so {:my-param true} has no context in relation to :checklists—it’s just along for the ride, wherever it might be needed.#2022-11-2914:40kendall.buchananThis is all very helpful, thank you.#2022-11-2914:40wilkerlucioit is in the context of :checklists subquery, and will only be available there (attributes in that vector)#2022-11-2914:40wilkerluciobut it gets detached from the :checklists attribute#2022-11-2914:41kendall.buchananOkay, yeah, better.#2022-11-2914:42kendall.buchananK, thanks @wilkerlucio, plenty of options to work with here.#2022-11-0420:35kendall.buchananI have the following [{:org.benefit/groups [:group/id :group/name]}]#2022-11-0420:36kendall.buchananBut an intermediate resolver (`:groups/children`) can take a param (`:public-mode? true`). What’s the right EQL syntax to pass it to the intermediate resolver?#2022-11-0420:37kendall.buchananThis appears to work, but it also returns additional results in the response map (which are not wanted or needed):#2022-11-0420:37kendall.buchanan
[(:group/children {:public-mode? true})
 {:org.benefit/groups [:group/id :group/name]}]
#2022-11-0517:10prncEDIT: Solved I’m trying to use Pathom Viz on mac (macbook air m1, macos 12.4), I’ve downloaded the Pathom-Viz-2022.8.21-mac from GH releases, and trying to run it I get
The application "Pathom " can't be opened.
dialog Running from terminal
open Pathom\ 
I get
The application cannot be opened for an unexpected reason, error=Error Domain=RBSRequestErrorDomain Code=5 "Launch failed." UserInfo={NSLocalizedFailureReason=Launch failed., NSUnderlyingError=0x600003290990 {Error Domain=NSPOSIXErrorDomain Code=111 "Unknown error: 111" UserInfo={NSLocalizedDescription=Launchd job spawn failed}}}
has anyone come across those difficulties in running Pathom Viz on mac before?
#2022-11-0517:15prncHa! the Pathom-Viz-2022.8.21.dmg OTOH seems to work šŸ™‚#2022-11-0913:21JHi! It’s ok to have custom field in mutation config?
(pco/defmutation my-mutation [env params]
  {::my-custom-key "value"}
  (...))
#2022-11-0913:38wilkerlucioyes, the config map is open, its design to allow for user extra keys for any reason you might wanna put there#2022-11-0913:38wilkerlucioa common example is to leverage those keys in a plugin implementation to configure some behavior#2022-11-0913:42JPerfect! Thanks šŸ‘#2022-11-1013:34Mark WardleHi all. Experimenting with union resolvers and queries. I've made it work using a nested property but think that this nesting is redundant, and wondered whether I could get it to work at the top-level.#2022-11-1013:37Mark WardleHere is a resolver that works:
(pco/defresolver component-by-id
  "Resolve an arbitrary SNOMED URI as per "
  [{::keys [svc]} {:info.snomed/keys [id]}]
  {::pco/output [{:info.snomed/component {:info.snomed.Concept/id [:info.snomed.Concept/id]
                                         :info.snomed.Description/id [:info.snomed.Description/id]
                                         :info.snomed.Relationship/id [:info.snomed.Relationship/id]
                                         :info.snomed.RefsetItem/id [:info.snomed.RefsetItem/id]}}]}
  {:info.snomed/component
   (cond
     (number? id) (case (snomed/identifier->type id)
                    :info.snomed/Concept {:info.snomed.Concept/id id}
                    :info.snomed/Description {:info.snomed.Description/id id}
                    :info.snomed/Relationship {:info.snomed.Relationship/id id}
                    nil)
     (uuid? id) {:info.snomed.RefsetItem/id id}
     (string? id) {:info.snomed.RefsetItem/id (parse-uuid id)})})
I can pass in an arbitrary :info.snomed/id and the identifier is parsed, the type determined, and a concrete result returned in the property :info.snomed/component. This means that this query returns the correct value:
(p.eql/process registry
               {:info.snomed/id "7c0d7d61-c571-5bf9-9329-fdbfee8747d0"}
               [{:info.snomed/component
                 {:info.snomed.Concept/id     [{:info.snomed.Concept/preferredDescription [:info.snomed.Description/term]}]
                  :info.snomed.Description/id [:info.snomed.Description/term]
                  :info.snomed.Relationship/id [:info.snomed.Relationship/id]
                  :info.snomed.RefsetItem/id [:info.snomed.RefsetItem/id :info.snomed.RefsetItem/referencedComponentId]}}])
Result: #:info.snomed{:component #:info.snomed.RefsetItem{:id #uuid"7c0d7d61-c571-5bf9-9329-fdbfee8747d0",:referencedComponentId 123558018}}
#2022-11-1013:39Mark WardleBut is it really necessary to nest this union within a property :info.snomed/component? I have tried to do this union query without nesting within that property, but get a result that looks more like EQL back: (p.eql/process registry {:info.snomed/id 80146002} {:info.snomed.Concept/id [:info.snomed.Concept/id {:info.snomed.Concept/preferredDescription [:info.snomed.Description/term]}] :info.snomed.Description/id [:info.snomed.Description/id]}) => {[:info.snomed.Concept/id [:info.snomed.Concept/id #:info.snomed.Concept{:preferredDescription [:info.snomed.Description/term]}]] #:info.snomed.Concept{:id [:info.snomed.Concept/id #:info.snomed.Concept{:preferredDescription [:info.snomed.Description/term]}]}, [:info.snomed.Description/id [:info.snomed.Description/id]] #:info.snomed.Description{:id [:info.snomed.Description/id]}}#2022-11-1013:40Mark WardleAlternative is to ignore union queries, and request all properties and only return the valid ones for that type?#2022-11-1121:13dvingoThis doesn't look like the typical union query setup - where you have a heterogeneous collection of things and each member of the collection has a unique attribute that is used for its id property. In your example, the value of the id attribute is used to determine the polymorphism instead#2022-11-1121:14dvingoyou probably already saw, but just for completeness https://pathom3.wsscode.com/docs/eql#union-queries describes the typical problem#2022-11-1121:22Mark WardleThanks @U051V5LLP I think it is a job for a union query because resolving on :info.snomed/id could result in a concept, description, relationship or refset-item. We can determine type based on the ID and its penultimate 2 digits, or if a UUID or string. It works, but only as a nested property, not a more flattened structure. #2022-11-1121:46Mark WardleBut maybe I should just accept some attributes will be unreachable depending on type, and always use lenient mode. But I thought unions were designed to solve this. Nesting isn’t so bad but I wonder whether I am either not declaring a top level union, or using the right query if indeed a top level union is possible. #2022-11-1121:58dvingoBut the branching in your case is on the value of the ID attribute, not the name of the Id attribute - that's how it differs from the usual union use-case. Does that make sense? My understanding of what you want to achieve is that yes, the resolver will declare all possible fields and for some of those entities certain fields won't be available#2022-11-1122:45Mark WardleYes that does make sense now. Thank you. I will make it simpler! #2022-11-1211:33Mark WardleSo if I am strict mode, the recommended approach is to make all attributes in my query optional? ie [(pco/? :info.snomed.Concept/id) ...] , or using lenient mode and ignoring unresolved properties, rather than using a union query which necessitates a nested property?#2022-11-1315:49dvingoI am not sure about what is recommended, but I think either strict or lenient would work - then it's up to the consumer of the data to detect the available fields#2022-11-1319:55Mark WardleThanks. This is working, even though for my use case, it forces lenient mode or a lot of declaring attributes to all be optional. I guess what I was looking for was a union-like thing in which I can declare multiple sets of attributes that can be provided depending on the request at hand, but at the top-level. Thank you for your advice.#2022-11-1014:26Eric Dvorsak@caleb.macdonaldblack what is the result you were expecting with this query? https://github.com/wilkerlucio/pathom3/issues/168#2022-11-1016:30Eric Dvorsakthe issue definitely seems to be coming from the index-io#2022-11-1016:30Eric Dvorsaksince it doesn't ref nested attributes it looks like this:
:index-io
                                      {#{} {:parent #:child{:id {}}},
                                       #{:parent} {:child {}},
                                       #{:child/id} {:child {}}}
#2022-11-1018:27caleb.macdonaldblackI’m expecting the result of parent-child with :child/id to be used to resolve the value of child-by-id and then that to be resolved by parent->child. #2022-11-1019:01Eric DvorsakYeah that's what I figured and the planner is stucked in a recursive loop on the parent->child resolver#2022-11-1021:48caleb.macdonaldblack@U03K8V573EC Do you think it’s related to https://github.com/wilkerlucio/pathom3/issues/167?#2022-11-1021:50caleb.macdonaldblackAlso afaik, this function is used to keep track of cycles and stop https://github.com/wilkerlucio/pathom3/blob/6c08dee47dafe5b3bf3a139056d2c6cb61613a73/src/main/com/wsscode/pathom3/connect/planner.cljc#L1404#2022-11-1021:51caleb.macdonaldblackSo it should be getting picked up in there, but isn’t. #2022-11-1114:26Eric DvorsakWould be interesting to make functioning resolvers for this example, I'm not sure what child value could be in this scenario#2022-11-1023:50sheluchinDoes Pathom have anything for dealing with data-driven assignment?#2022-11-1101:34dehliwhat do you mean by data-driven assignment?#2022-11-1110:33sheluchinThat's when data is structured using some of the data as the key.
[{:person/first-name "linus"
  :person/last-name "torvalds"
  :person/famous-for "linux"}
 {:person/first-name "rich"
  :person/last-name "hickey"
  :person/famous-for "clojure"}]

{"linus" {:person/last-name "torvalds"
          :person/famous-for "linux"}
 "rich" {:person/last-name "hickey"
         :person/famous-for "clojure"}}
#2022-11-1112:47wilkerlucio@UPWHQK562 can you tell more on what you are trying to do? this map looks a simple lookup by name#2022-11-1119:23sheluchin@U066U8JQJ I don't have a particular application for it right now but it is a fairly common pattern that some REST APIs return their data in. It is a simple lookup by name, except it could be any number of keys and you may not know them. I recall having to massage some data that was given to me like this to make it work well with Pathom, and I was just wondering if this is a common enough pattern that Pathom has some utils for dealing with it.#2022-11-1119:25wilkerlucioyou can check the built in resolvers, there are a couple helpers to deal with this format, but mostly for static data (vs dynamic data via api), but it can be a source of inspiration for you to implement dynamic ones#2022-11-1119:26sheluchinThank you.#2022-11-1201:15wilkerluciothe format you sent matches the attribute tables helper: https://pathom3.wsscode.com/docs/built-in-resolvers#attribute-tables#2022-11-1512:46sheluchinThanks, @U066U8JQJ. That connection was not obvious to me, but I think I understand it now.#2022-11-1118:19David Yanghey all - I have a resolver that describes a deeply nested output structure but I’m unable to join against the keys in that nested output here’s the gist: https://gist.github.com/davidyang/b492acf0024f480139eadf79bc5e214a#2022-11-1118:21dehliDid you specify ::pc/output using nested output? Ex: {:my/output [:nested/a :nested/b]}#2022-11-1118:22David Yanghey @U2U78HT5G - yes I specified the ::pco/output option#2022-11-1118:24David Yanghmmm, this seems to work correctly:#2022-11-1118:24David Yang
(p.eql/process env [{[:user/id 1] [:user/id
                                     :user/full-name
                                     {'(:user/feed {:start-date 1 :end-date 2})
                                      [{:feed/day 
                                        [:feed/date
                                         {:feed/user-days 
                                          [:feed/food :user/id :user/full-name]}]}]
                                      }]}])
#2022-11-1118:24David Yangto have them at the same level of nesting - just the joining doesn’t seem to work#2022-11-1118:26David Yangbut this doesnt:
(p.eql/process env [{[:user/id 1] [:user/id
                                     :user/full-name
                                     {'(:user/feed {:start-date 1 :end-date 2})
                                      [{:feed/day 
                                        [:feed/date
                                         {:feed/user-days 
                                          [:feed/food {:user/id [:user/full-name]}]}]}]
                                      }]}])
#2022-11-1118:35dehliThe problem with your second query is that you're asking for :user/full-name underneath :user/id which doesn't make sense since :user/id is a string. If instead you made the response look like:
:feed/user-days [{:feed/user {:user/id 2}
                  :feed/date ....
You could then ask for [:feed/food {:feed/user [:user/id :user/full-name]}]
#2022-11-1118:36dehlior your first query is another valid way to ask for the data#2022-11-1118:36dehligenerally it's better to keep pathom data flatter as it gives your resolvers more input data they can use#2022-11-1118:38David Yangthanks for the help - I don’t understand what you mean when you say that :user/id is a string?#2022-11-1118:39dehlioops, sorry in your example it was a number#2022-11-1118:39dehlibut the main point is that it's not an object#2022-11-1118:39dehlinested queries only make sense when the response is an object#2022-11-1118:39David Yangmy mental model is that the returned attributes can be converted to their references, but it sounds like they need to return idents (the map of {:user/id 2})#2022-11-1118:40David Yangso if I have an ident as the value of the map then I can join on that#2022-11-1118:40dehlithat is true, which is why you can ask for both :feed/date and :user/full-name#2022-11-1118:40David Yangand when I ask for :user/full-name it seems to be smart enough to use the user/id at it’s sibling level or I guess whatever is lower in the tree#2022-11-1118:41dehliit would be sibling, pathom won't look for attributes lower in the tree#2022-11-1118:43David Yangso pathom doesn’t go up the tree looking for idents, it needs it at the sibling level#2022-11-1118:43dehliThe only reason you would include :feed/user {:user/id 1} is if you needed the user attributes to be in an isolated key For example, the response to the query would be:
:feed/food "food"
:feed/user {:user/id 1 
            :user/full-name "Rich"}
but if you don't need the user attributes to be in their own key, it's preferable to have all the attributes at the same level which means your response would look like:
:feed/food "food"
:user/full-name "Rich"
#2022-11-1118:44David Yangok - so just rely on pathom’s smart resolver to look for a sibling when I join on new attributes#2022-11-1118:44dehlicorrect, in the example you posted you had :user/id as a sibling to :feed/food which is why you could also for :user/full-name at the same level.#2022-11-1118:44David Yangso I join on user/full-name at that level it’ll look for the :user/id at that level (that level only right?) and resolve it that way#2022-11-1118:44David Yangthat’s very cool#2022-11-1118:45dehliyep! and if it already has that resolver response cached it won't need to do another lookup (say that resolver required a database lookup)#2022-11-1118:45David Yangbut that means that if I need a date parameter I need to pass it down the tree - I can’t rely on it to find the attribute from higher in tree#2022-11-1118:48dehliPotentially, if you're able to keep your resolvers pretty flat you might not need to pass the data down. It also might make more sense as input depending on how it's being used#2022-11-1118:51David Yangis there a performant way to break out these nested resolvers? maybe but a look-through cache in env so that when the nested resolvers are called they can get the data from the cache instead of having to fetch it again from DB? Can I separate out the resolvers from the data nesting itself without multiplying the calls out?#2022-11-1118:56dehliThere are ways to do that. One thing I've done before is have a top-level resolver that does one query to get all needed data and then hydrates pathom's resolver cache so that downstream resolver calls will already have the data accessible. Pathom has an optional parallel parser that will parallelize all the requests it can, which might be enough for what you need.#2022-11-1118:59David Yanginteresting - so if I do this big resolve it’ll normalize it for subsequent queries?#2022-11-1119:01dehliYou could (doesn't do that out of the box). I was able to get it working by creating one resolver that would interface with my db and then I just had to normalize to that db resolver#2022-11-1119:02dehliThere's also batch resolvers (https://pathom3.wsscode.com/docs/resolvers/#batch-resolvers)#2022-11-1119:02David YangI guess to clarify - if I have a query that returns a deeply nested structure, but I define only the top level outputs, if I have separate resolvers that access that nested structure pathom could figure it out?#2022-11-1119:02David YangOr are you saying I would have that top level resolver put it’s substructure into the resolver cache somehow#2022-11-1119:03dehliIf you return a deeply nested structure, downstream resolvers would be able to access all the nested data. Every "nested level" is a new context though which means attributes above or below wouldn't be accessible (just that level itself)#2022-11-1119:05dehlibut your query would also need to closely match the response (which takes away from the value of pathom)#2022-11-1119:05David Yangand I’d have to define the output with all the nesting correct?#2022-11-1119:06dehlicorrect#2022-11-1119:07David YangOK - that makes sense to me - the batching is cooler than I first understood from reading, it sounds like it aggregates many places in the tree (so it can combine multiple queries for comments and then I can group those into one SQL query)#2022-11-1119:08David Yangfor some reason I thought it was just passing in a sequence of items from one level to another - not all the requests from the different parts of the EQL tree#2022-11-1119:09dehliyeah, I'm not 100% sure tbh. i haven't used batch very much#2022-11-1119:10David YangOK - lots to experiment with. thanks so much for your help! very much appreciate your time/insight#2022-11-1119:11dehlihappy to help! šŸ™‚ feel free to post again if you have any other questions!#2022-11-1120:50caleb.macdonaldblack@wilkerlucio From my own experience and perusing the slack history, GitHub issues and documentation, Pathom3 seems to stuggle with solving cyclic problems among others like resolving back up a nested datastructure (maybe also cyclic?). I lack the experience, but could you provide any insight into this? Is there something fundamental wrong with trying to solve these cyclic problems in this paradigm? Perhaps it’s impossible? Or maybe it’s a design choice to avoid solving problems this way? For example Clojure struggles with cyclic dependency resolution and community consensus is that good design shouldn’t need cyclic dependencies. (Apologies for no references). If none of those, maybe this functionality just isn’t here yet? I’m eager to dive into these interesting problems but with my lack of experience in Pathom, I just hope I’m not wasting my time solving the unsolvable. I could put together some use-cases if anything needs clarity.#2022-11-1616:13caleb.macdonaldblack@UPWHQK562 #2022-11-1619:03wilkerluciohello @U3XCG2GBZ, I love if you can provide some use cases so I have a better understanding of the problem#2022-11-1619:04wilkerlucioin general, Pathom has cycle detection and that will case a failure in planning, since there is no way to resolve the cycle#2022-11-1803:55caleb.macdonaldblack@wilkerlucio Thanks for getting back to me on this one. https://github.com/wilkerlucio/pathom3/issues/168 I am running into a stack overflow with the example in that issue. If the cycle detection was working I believe it would log a warning and stop the planner from cycling here. Is there something fundamentally wrong with the approach here in the example? Perhaps the cyclic warning log is a clue of that? #2022-11-1812:48wilkerlucioI started looking at it yesterday, I was able make your example work, but it also made a test fail, so Im trying to get all working, its in progress ;)#2022-11-1905:11caleb.macdonaldblackOh thanks. Would you mind pushing your WIP changes? I would like to take a look if I could. #2022-11-1123:31David YangHey - I’m planning on running the results of my EQL queries through normalization - but if I use put references to :user/id in other objects then the normalizer will percieve that as a user object. Is the solution to put something like :feed/user-id in that object and then put an alias in the resolver chain?#2022-11-1408:42Tom H.If I put a custom field in a resolver config like so:
(pco/defresolver my-resolver [env params]
  {::permissions/require #{:project-user}}
  (...))
And then use ::pcr/wrap-resolve in a plugin like so:
(defn protect-resolver-wrapper [resolve]
  (fn [env input]
    (tap> env)
    (resolve env input)))

(p.plugin/defplugin protect-resolver-plugin
  {::pcr/wrap-resolve protect-resolver-wrapper})
Am I able to check for my custom key? I can’t seem to spot it in the env or input except in the indexes. Should I look up the resolver in the indexes using another key?
#2022-11-1412:21PanelDoes anyone use pathom without fulcro ? #2022-11-1412:26markaddlemanWe do. We use Pathom as an API layer for an a vue.js application.#2022-11-1412:28Tom H.Yup! Pathom (via Pedestal) serving re-frame clients#2022-11-1412:51PanelSo you just send eql query or using something to sync the front end ?#2022-11-1413:12Tom H.Many of our re-frame events still do a regular http request to various backend endpoints but we’re slowly moving everything to a single /pathom POST endpoint. We use Pedestal interceptors to do some authentication, insert db connections and some other things then the requests are resolved using a com.wsscode.pathom3.interface.eql/boundary-interface that’s constructed on the fly for each request. We pass the pedestal context into the pathom environment so resolvers have access to the database and authentication etc.#2022-11-1413:14Tom H.It’s up to the re-frame event that sent the request to deal with the result in some way but we’re working on a more systematic approach#2022-11-1414:28danierouxFulcro-RAD with Datomic Cloud and Pathom3. ... it is such a joyful place to be.#2022-11-1415:20jmayaalvwe use pathom as an api as well, with a graphql engine on top for a part of the index#2022-11-1419:05Mark WardleYes definitely. Have a re-frame front-end talking to pathom via pedestal - have functions in front-end just send the EQL and then generally, but not always, normalise into the state db. But I then moved to Fulcro and a lot of the code I needed to write was no longer needed.#2022-11-1422:53PanelCan anyone share some public code ? #2022-11-1514:08Mark WardleI wouldn't do it quite like this now, but here is an example https://github.com/wardle/pc4/blob/main/pc4-ward/src/eldrix/pc4_ward/snomed/events.cljs - I have events in a number of different namespaces correlating with domains within healthcare - so this one is SNOMED CT - you can see I simply define EQL at the top level and then pass those data down my react tree. Fulcro takes a different approach, which has been easier in many respects. If I were doing this again, I'd normalise my results using something like 'pyramid' perhaps.#2022-11-1514:10Mark WardleArguably, for simple demos, re-frame and bespoke EQL is easier, but I would have needed to re-architect if I wanted to move to anything less trivial, and likely reimplement the stuff one gets in Fulcro for free.#2022-11-1518:30dvingoI have been frustrated with using fulcro for UI rendering, but it does have great features for mutations and networking, speaking to pathom. https://github.com/matterandvoid-space/todomvc-fulcro-subscriptions I put this demo app together showing how you can use subscriptions (from re-frame) and helix to render your UI while using fulcro to talk to pathom and manage your app-db, and managing form state. It does require some fulcro knowledge, but only the data manipulation part. In my experience the complexity of fulcro is on the UI side (defsc, composing queries, the router). Getting rid of the UI part has made developing with fulcro a lot easier for me. I'm not sure what your motivation for not wanting to use fulcro is but maybe this can inspire some ideas.#2022-11-1517:57sheluchinCan someone explain to me why this doesn't work (snippet in thread)? I have items and sub-items. My sub-items contain pointers (`:sub-item/parent`) to their parent, but it seems that Pathom isn't able to follow the relation, even though all the attributes are declared. Is this sort of operation not supported because the item -> sub-item -> item puts a cycle in the DAG?#2022-11-1517:58sheluchin#2022-11-1616:12caleb.macdonaldblackInterested in knowing more about this also. #2022-11-1616:12caleb.macdonaldblack@UPWHQK562 I’m having struggles with similar examples. #2022-11-1622:05Ben Grabow
{::pco/output [:sub-item/id
                 {:sub-item/parent [:item/id]}]}
This output spec claims that the resolver will return a 1-to-many relationship from 1 :sub-item/parent to many records that each have an :item/id, but that's not what the resolver returns. It returns data with a 1-to-1 relationship between :sub-item/parent and :item/id.
#2022-11-1622:06Ben GrabowYou may want to model the parent ID as a top-level attr of the sub-item record:
(pco/defresolver sub-item
  [{:sub-item/keys [id]}]
  {::pco/output [:sub-item/id :item/id]}
  (let [sub-items {1 {:sub-item/id 1 :item/id 1}
                   2 {:sub-item/id 2 :item/id 2}}]
    (get sub-items id)))
#2022-11-1622:11Ben GrabowThe other option is to model the :sub-item/parent attribute as a vector of parents rather than a singular parent record (note the [] around {:item/id 1} and {:item/id 2}:
(pco/defresolver sub-item
  [{:sub-item/keys [id]}]
  {::pco/output [:sub-item/id
                 {:sub-item/parent [:item/id]}]}
  (let [sub-items {1 {:sub-item/id 1 :sub-item/parent [{:item/id 1}]}
                   2 {:sub-item/id 2 :sub-item/parent [{:item/id 2}]}}]
    (get sub-items id)))
#2022-11-1623:06sheluchin@UANMXF34G my understanding is that one of the limitations of EQL is that it doesn't specify whether the join is to a single value or a collection, and that {:x [:y]} is to-one or to-many depending on what is found. From Fulcro's docs: >Joins are automatically to-one if the data found in the state is a singular, and to-many if the data found is a vector. Is this a Fulcroism that doesn't apply to Pathom?#2022-11-1623:17Ben GrabowI don't know that much about Fulcro, so I'm not sure if this is just a Fulcro thing or a Pathom thing. It's surprising to me to hear that to-1 joins are possible with this syntax. However I see the EQL docs say the same thing! https://edn-query-language.org/eql/1.0.0/what-is-eql.html#_eql_for_selections Have you been able to traverse a to-1 join with Pathom before? I have never tried it, so it jumped out at me that you were trying it.#2022-11-1623:25sheluchinAfk right now so can't look at my code for exact examples, but I believe I have. 90% certain. Thank you for the suggestion but I believe it's inconclusive for now. I'll try to put some tests around this theory when I get in front of a computer. I would be surprised if it were the case, but it would be nice if a long held erroneous understanding got corrected.#2022-11-1623:30Ben GrabowI think I found the problem! Just a small typo:
(pco/defresolver item
 [{:item/keys [id]}]
 {::pco/output [[:item/id :item/foo]]}
 (get {1 {:item/id 1 :item/foo "a"}
       2 {:item/id 2 :item/foo "b"}}
      id))
should be
(pco/defresolver item
 [{:item/keys [id]}]
 {::pco/output [:item/id :item/foo]}
 (get {1 {:item/id 1 :item/foo "a"}
       2 {:item/id 2 :item/foo "b"}}
      id))
There were extra [] around the output spec.
#2022-11-1623:30Ben GrabowThe query works as written now:
(p.eql/process
  (pci/register [all-items
                 item
                 sub-item
                 earliest-sub-item])
  [{:all-items [:item/id
                ;; works!
                :item/foo
                {:item/earliest-sub-item [:sub-item/id
                                          {:sub-item/parent
                                           [:item/id :item/foo]}]}]}])
=>
{:all-items [#:item{:id 1, :foo "a", :earliest-sub-item #:sub-item{:id 1, :parent #:item{:id 1, :foo "a"}}}
             #:item{:id 2, :foo "b", :earliest-sub-item #:sub-item{:id 2, :parent #:item{:id 2, :foo "b"}}}]}
#2022-11-1623:34Ben GrabowIt was my own misunderstanding. Today I learned that we can express to-one queries in EQL!#2022-11-1623:48sheluchinHah, oh man... Well, thank you. I wish there was same way to catch typos like that. I was looking for a typo in a misspelled key. OK, so now to continue with the repro of my actual issue lol I'll start another thread for that :)#2022-11-1523:59rschmuklerIs there any way to get a non-batched value from a pathom batched resolver? Eg.
(defresolver foo-resolver
  [_]
  {:foo 5})

(defresolver users-resolver
  [_]
  {:users [{:user/id 1}]})

(defresolver batch-computed-foo-resolver
  [users]
  {::pco/input [:user/id :foo]
   ::pco/output [:user/foo-times-two]}
  (mapv (contantly (*2 (-> users first :foo))))
When I have something like the above I get an error saying that it can't find a path for :user/foo-times-two when I try and query for {:users [:user/id :user/foo-times-two]} - if I remove :foo from the input then it can find a path
#2022-11-1722:04wilkerlucioI think the issue is that your response has only the value, you need each item on the collection to be like {:user/foo-times-two 5}#2022-11-1722:05wilkerlucioif its not, please provide a repro and we can check, mixing batch and non batch should be fine#2022-11-1722:29rschmukler@U066U8JQJ Thanks for the reply! I posted a repro in the GitHub of the issue that I am facing. Please let me know you need any more information. I apologize the code example above was incorrect (but in my repro, and in prod, I am indeed returning the full maps, not just the value) Link to the issue: https://github.com/wilkerlucio/pathom3/issues/171#2022-11-1723:27wilkerlucioreplied at the issue#2022-11-1610:42roklenarcicThe client always requests certain attributes (like :tempids ) but often none of the resolvers executed return the attribute, which leads to `
::p.error/attribute-unreachable
a lot of the time. Is there a way around this? I wanted to introduce a resolver that returns :tempids {} without any input. But what will happen when the mutation DOES return :tempids {….}? which will be returned to the client?
#2022-11-1612:21markaddlemanhttps://pathom3.wsscode.com/docs/resolvers/#prioritization might help#2022-11-1616:17caleb.macdonaldblackPriority should help here. I played around with a similar idea of a resolver that would generate and return ids if one wasn’t provided but it was a bit too confusing to debug. I recall caching having an interesting role. If I wanted multiple new ids, my resolver would return the initial cached value. I ended up scrapping the idea before messing with the caching. #2022-11-1616:18caleb.macdonaldblackIs it possible the client could optionally request tempids? (pco/? :tempids)#2022-11-1616:19roklenarcicDoes (pco/? :tempids) work over the wire?#2022-11-1616:19roklenarcicusing transit here#2022-11-1616:20roklenarcicthe issue is that there are many transformations happening between my client code and server that transform query->ast->query again#2022-11-1616:22roklenarcicoh so it’s a param#2022-11-1616:22roklenarcicthat should work I think#2022-11-1616:30caleb.macdonaldblackI recall it working in an eql query#2022-11-1616:32markaddlemanSomething like (pco/? :tempids) operates at the Pathom planner level so it is internal to how Pathom decides when there is sufficient input for a resolver#2022-11-1722:02PanelI mapped a legacy system to be queried with pathom, at some point it will be decommissioned. How could I use pathom to ingest the data into a db ?#2022-11-1722:06wilkerlucioyou need to first figure the structure of the database you are sending the data too, then you can try to make some queries for each entry, in a way you get the data you need to send to the database#2022-11-1802:02PanelAre there built-in helper to throttling when consume API in resolver ?#2022-11-1822:51wilkerlucionothing included in Pathom#2022-11-2109:25Eric Dvorsakyou want to throttle globally or per resolver?#2022-11-2109:27Eric DvorsakI've implemented per-resolver throttling in Lacinia in the past by wrapping resolvers and using redis. if you only have one instance of your application you could use core cache with a TTLCacheQ and wrap the resolvers you want to throttle#2022-11-2111:05ParmijeanneHello, I'm trying to consume a graphql endpoint but I'm a bi lost when it come to writing to eql for it. Here's the request I'm trying to replicate:
{
   "operationName":"SearchQuery",
   "variables":{
      "input":{
         "q":"*Eucalyptus*",
         "rows":50,
         "fq":[
            
         ],
         "page":1,
         "facetLimit":20,
         "facetField":[
            "taxonomic_status",
            "occurrence_status",
            "establishment_means",
            "degree_of_establishment",
            "taxon_rank",
            "family"
         ]
      }
   },
   "query":"query SearchQuery($input: SearchInput!) {
  search(input: $input) {
    docs {
      id
      taxonRank
      acceptedNameUsage
      acceptedNameUsageId
      acceptedNameUsageAuthorship
      preferredVernacularName
      scientificName
      scientificNameAuthorship
      family
      taxonomicStatus
      nameAccordingTo
      __typename
    }
    meta {
      params {
        q
        fq
        fl
        rows
        __typename
      }
      pagination {
        lastPage
        total
        currentPage
        __typename
      }
      __typename
    }
    facetFields {
      fieldName
      fieldLabel
      facets {
        value
        count
        fq
        __typename
      }
      __typename
    }
    __typename
  }
}"
}
That's the eql I'm trying to send so far... I just don't understand how to pass the "nested" params to the query.
[{(:herb.Query/search  {:herb.SearchParameters/q "*Euca*"})
  [{:herb.SearchResult/docs [:herb.SearchResultDocument/id]}]}]
The public endpoint is ""
#2022-11-2111:14souenzzosomething like this:
[{`(~'SearchQuery {:q          "*Eucalyptus*"
                   :facetField ["taxonomic_status"]})
  [{:docs [:id :taxonRank :__typename]}
   {:meta [{:params [:q]}
           {:pagination [:lastPage]}
           :__typename]}
   {:facetFields [:fieldName
                  {:facets [:value]}
                  :__typename]}
   :__typename]}]
#2022-11-2111:50ParmijeanneThanks mate, I got it working, thanks to pathom viz autocomplete and lots of try and error...
[{(:herb.Query/search {:input {:q          "*Eucalyptus*"
                               :facetField ["taxonomic_status"]}})
  [{:herb.SearchResult/docs [:herb.SearchResultDocument/id]}]}]
#2022-11-2121:12PanelCan you use wildcard when consuming graphql ?#2022-11-2121:14wilkerlucioyes you can, but its important to understand what wildcard means. in Pathom, the wildcard means "expose all the data that was gathered for this entity during processing", and not "get me everything possible"#2022-11-2121:14wilkerluciothis means that if you have something that depends on a graphql thing, and add the wildcard, you will see that data there, but Pathom will not ask for anything that's not part of (or a dependency of) your original query#2022-11-2121:17PanelThanks for making it clear, I was definitely expecting the « get me everything » scenario. #2022-11-2121:17Eric DvorsakI'm getting bit hard by aliases now, pathom is finding all kind of paths that are semantically incorrect for users, as I have a lot of one to one and one to many relationships involving user/id. I guess the remedy is to not keep things too flat after all#2022-11-2121:18wilkerlucioits tradeoff, but in general is better to avoid multiple paths if you have a preferred one#2022-11-2121:18wilkerlucioreducing OR nodes (by reducing the number of paths for a given attribute) in the planner will make things run smoother#2022-11-2121:23Eric DvorsakThese are accidental (and wrong). I'll try to make a minimal example tomorrow. For instance trying to get the users that are in a course by course/id, one of the path in the OR node will do it through purchases by course/id since it outputs purchase.user/id aliased to user/id#2022-11-2121:23Eric DvorsakSo it's semantically wrong since it's only the users that purchased a course not all the users in the course#2022-11-2121:33wilkerlucioyeah, its an important detail to be aware, if you have ambiguity in the path, better to avoid it, but I think its an important detail that we should talk more about it when flatening things#2022-11-2121:33wilkerlucioits good, but only when there is no ambiguity#2022-11-2121:33wilkerlucioif there is, it becomes a tricky land#2022-11-2121:35wilkerluciolets think about the courses thing#2022-11-2121:35wilkerluciomy idea there would be to implement :course/users for the users of a course, that depends on :course/id, whats the way you have it?#2022-11-2202:05PanelCan I reuse the response from one query into another one in the same eql ? In the following eql I have multiple :herb.SearchResultDocument/id returned from the first search, I want to use the value into another search for each id, mapped as :taxonConceptId
[{'(:herb.Query/search {:input {:q          "*Eucalyptus*"
                                     :facetField ["taxonomic_status"]}})
       [{:herb.SearchResult/docs [:herb.SearchResultDocument/id  <-- USE THIS
                                  :herb.SearchResultDocument/scientificName
                                  :herb.SearchResultDocument/acceptedNameUsage
                                  :herb.SearchResultDocument/preferredVernacularName]}]}
 {'(:herb.Query/taxonConceptImages {:taxonConceptId "22ad8546-8f00-4999-9b20-e40d11229ad5" <-- HERE })
       [{:herb.ImagePaginator/data [:herb.Image/previewUrl]}]}]
#2022-11-2206:20PanelI can't see any example of resolved paras... I'm probably missing something, or I'm just supposed to do multiple queries. Not sure you can have ā€œlogic variablesā€in eql#2022-11-2410:28Mark WardleHi. If I am understanding your requirement correctly, you could create a resolver to alias :herb.SearchResultDocument/id as :taxonConceptId - and then resolve any dependent properties from there, when possible. So you actually only make one request, but it is satisfied by a number of different resolvers. If your backing lookup service can resolve multiple at the same time, then you could use batch resolvers to optimise the multiple fetches required.#2022-11-2410:30Mark WardleIf there is a 1:1 map between your search result document id and the taxon concept id, then you'd ideally want to flatten these resolvers at the top-level. For other requirements, you might nest the properties. I have resolvers that coordinate across multiple backend services in a single request using the results from one resolution in the processing of other requested attributes.#2022-11-2410:36Mark WardleYour alternative is to add a resolver that can take a :herb.SearchResultDocument/id and output a :herb.Query/taxonConceptImages attribute - or you could rename and abstract the how and think in terms of the API you wish to provide to clients, and simply resolve :herb.searchResultDocument/images and then have nested attributes that make sense in that context.#2022-11-2500:41PanelThanks for the explanation, I was using he dynamic resolver or graphql and just dropped that and wrote my own resolver that post json to a graphql endpoint.#2022-11-2507:30Mark WardleHi. I wouldn’t necessarily remove dynamic resolvers, but simply use the first option I mentioned and alias one attribute for another. When pathom knows it can get from your search result id to a taxon concept id and using that can get all other requested attributes, it should work. #2022-11-2402:33pfeodrippe[RESOLVED] Hi =D In Pathom 3, do we have a built-in trace plugin? If not, it seems that the plugin system is powerful enough that I could try it \o#2022-11-2402:48pfeodrippeHunnn, guess I should have everything I need on ::pcr/run-stats? Will try it#2022-11-2409:38Eric Dvorsakwhat kind of trace are you expecting? If you use pathom-viz you have a visualization of your query planning/running#2022-11-2413:37pfeodrippeI don’t want to visualize it, but parse it to text so I can show it in Clerk. But the run stats seem to do the job, thanks :)#2022-11-2505:13pfeodrippe[RESOLVED] Hi o/ I am trying to see which attributes a node is trying to resolve, I've thought that I could rely on ::pcp/source-for-attrs which is used at https://github.com/wilkerlucio/pathom3/blob/main/src/main/com/wsscode/pathom3/connect/runner/stats.cljc#L72, but I don't see it in my run-stats response for a query (see below for an example). Is this expected? Maybe is there another way to see the current attribute a node is trying to resolve?
{... ...

 :com.wsscode.pathom3.connect.planner/nodes
 {20
  {:com.wsscode.pathom3.connect.operation/op-name my.resolver/a->b
   :com.wsscode.pathom3.connect.planner/expects #:db-my.person{:id {}}
   :com.wsscode.pathom3.connect.planner/input #:my.person{:external-id {}}
   :com.wsscode.pathom3.connect.planner/node-id 20
   :com.wsscode.pathom3.connect.planner/node-parents #{21}
   :com.wsscode.pathom3.connect.planner/run-next 16},

  ... ...}}
#2022-11-2513:07Eric Dvorsakthe attributes a node is trying to resolve are in :com.wsscode.pathom3.connect.planner/expects so in your case #:db-my.person{:id {}}#2022-11-2513:11pfeodrippeHunnnn, I’ve thought this was the return of the resolver, but you are right, Eric! Thanks :D Wonder if source-for-attrs is really used anymore #2022-11-2710:25nivekuildoes anyone else struggle with having to deal with both {:person/house {:house/id 1}} and {:person/house 1} in their code? like you probably store :person/house as an integer in your database right, but do you use it as a map everywhere else in your code?#2022-11-2714:49wilkerluciothis is overloading the semantics of person/house, which is a bad thing, my suggestion is to make the id field be person/house-id, so they have different names#2022-11-2720:57nivekuiloh... I may have some schema changes to run#2022-11-2710:28nivekuiland what happens if you decide it's ok as an integer at first, but then you want to promote it to its own type/ident?#2022-12-0413:59Ben GrabowI typically follow the rule of "give different things different names". Instead of reusing the same name for a new type of information, I create a new name for the new thing. For example, without being creative with the names, I might do something like this for your example:
;; Initial schema
{:person/house 1}

;; Accrete a new attribute to the schema
{:person/house 1
 :person/house-map {:house/id 1}}
#2022-11-2820:56c0rrzinis there a way to unregister a mutation or resolver from an index?#2022-11-2820:58c0rrzinthere involves a public api - I really didn't want to mess with the index map myself#2022-11-2915:03caleb.macdonaldblackI’m not sure myself but I’m intrigued. What’s the usecase? #2022-11-3020:11wilkerlucioits not trivial because the way the index is build, some indexes are accumulative and its hard to know if something was add multiple times by different resolvers/mutations, in that case it would require some full analysis to avoid removing something that might break the index#2022-11-3020:11wilkerluciothe simplest way is to re-build the index without the things you dont want#2022-11-3020:11wilkerlucioyou can use the index-resolvers & index-mutations to do so, get everything there, remove what you dont want, and rebuild the index#2022-11-3020:13c0rrzinalright, thanks!#2022-11-3020:16c0rrzin@U3XCG2GBZ the use case is that I want to dynamically register user mutations and resolvers, and the final index should change given the context of that user#2022-11-3011:02pieterbreedHi šŸ‘‹:skin-tone-4: - I'm maintaining some code that's been around from earlier in pathom3 version history. I have lenient mode on and intercept both mutation and resolver errors using the plugins. What's the official line on ::pcr/nodes-with-error and ::pcr/node-error meta-data? • Is this data still populated? (I see the plugin has been removed from docs) • Is there anything I can get by inspecting that meta-data that I can't get from the error intercepting plugins or by inspecting the error attributes on the result? • Is there any documentation on this meta-data? #2022-11-3020:12wilkerluciohello, yes, the data is populated, the plugin was removed because it became native (it always does that in lenient mode)#2022-11-3020:13wilkerluciodocs available at: https://pathom3.wsscode.com/docs/error-handling#2022-11-3020:13wilkerluciobut if its missing something you need, please let me know#2022-12-0108:41pieterbreedok thank you šŸ‘:skin-tone-4: I'm feeling a little dumb; I'm not seeing anything about meta-data in the error handling page?#2022-12-0114:05wilkerluciothe meta is used internally by the fns described in that page, you can see it in the attribute-error impl for example: https://github.com/wilkerlucio/pathom3/blob/main/src/main/com/wsscode/pathom3/error.cljc#L75#2022-12-0114:05wilkerlucioin this line it pulls the stats from the meta#2022-12-0115:10pieterbreedOk - what I’m hearing you say is this is part of the implementation. On our side I will remove the code that depended on it. Thank you, pathom is cool to work with#2022-12-0119:01kendall.buchananFun question: Is there such a notion as dispatching between resolvers based on the value of a key? For example, say you have :my/favorite-color which could have both :blue or :red values. In one resolver, you include :my/favorite-color as an input, but you only want to use this resolver if the value is :blue. Why? Because one resolver might require another input, like :my/favorite-season, in which case, you only want to access that key for :blue values, not :red values. (Perhaps for performance reasons—`:my/favorite-season` could be irrelevant for values of :red).#2022-12-0121:14wilkerluciono, that's not an notion for Pathom itself, planning doesn't consider anything on the data, this is important to allow proper caching, and also, this could make the whole processing much more complex to handle (from the. Pathom side). that said, there are ways to accomplish something like this, its totally valid to do a pathom request from inside a resolver, so here is one way you can do it:
(ns com.wsscode.pathom3.demos.dispatch-by-value
  (:require
    [com.wsscode.pathom3.connect.indexes :as pci]
    [com.wsscode.pathom3.connect.operation :as pco]
    [com.wsscode.pathom3.entity-tree :as p.ent]
    [com.wsscode.pathom3.interface.eql :as p.eql]))

(pco/defresolver favorite-season
  [{:keys [my/id]}]
  {:my/favorite-season (str "season for " id)})

(pco/defresolver favorite-thing
  [env {:keys [my/favorite-color]}]
  {:my/favorite-thing
   (case favorite-color
     :blue
     (let [season (p.eql/process-one env (p.ent/entity env) :my/favorite-season)]
       (str "with season " season))

     :red
     "go with red")})

(def env
  (pci/register
    [favorite-season
     favorite-thing]))

(comment
  (p.eql/process env
    {:folks
     [{:my/id             1
       :my/favorite-color :blue}
      {:my/id             2
       :my/favorite-color :red}]}
    [{:folks [:my/favorite-thing]}]))
#2022-12-0121:28kendall.buchananMmm, gotcha. I’ve often wondered about the advisability of invoking a Pathom query from within a resolver, and so far been able to avoid it.#2022-12-0121:29kendall.buchananBut I can understand the value of steering clear of dispatching on values.#2022-12-0200:26Tyler NisonoffHey! dealing with a perf issue in Pathom2 I have a resolver that has an input :foo/large-map
::pc/input [:foo/large-map]
It seems that pathom is spending a lot of time processing / doing something with the map despite its resolver looking something like
{:foo/large-map (with-meta large-map {::p/final true}) }
I see 2 seconds in between a log right before my return of the large-map above and the first line of the parent resolvers function was trying to follow the discussion https://clojurians-log.clojureverse.org/pathom/2021-02-12 to see if its relevant, but was having trouble following. Should my ::p/final meta here be circuiting pathom?
#2022-12-0213:04wilkerluciohello Tyler, it looks correct to me, and the final should short circuit (meaning pathom shouldn't try to process the items there), but usually a large map isn't an issue, most of the time a large sequence is an issue (because of processing a large N number of items)#2022-12-0213:04wilkerluciocan you make a repro of the issue that we can work on top?#2022-12-0216:11Tyler Nisonoffyeah, let me try to do that today! thanks wilker#2022-12-0217:16Tyler Nisonoffmy minimal repro doesnt have this behavior! new theory is perhaps a plugin in my fulcro project is doing something silly? 🤷#2022-12-0218:27wilkerlucioit might be a plugin to remove the special values (`p/elide-special-values` if I can remember correctly) from pathom (`::p/not-found` and ::p/error) from the output, this plugin needs to scan all the data, and might be causing your issue#2022-12-0218:27wilkerlucioif that's the case, you can write a version of the plugin that also looks for ::p/final on meta, and skip when its the case#2022-12-0220:06Tyler Nisonoffwas just coming here to say that appears to be whats happening, thank you!#2022-12-0220:47Tyler Nisonoff@U066U8JQJ got an implementation working with a modified transduce-maps:
(defn prewalk-with-final-check
  [f form]
  (if (::p/final (meta form))
    form
    (walk/walk (partial prewalk-with-final-check f) identity (f form))))

(defn transduce-maps-with-final-check
  "Walk the structure and transduce every map with xform."
  [xform input]
  (prewalk-with-final-check
    (fn elide-items-walk [x]
      (if (native-map? x)
        (with-meta (into {} xform x) (meta x))
        x))
    input))
Would you accept a PR for this in pathom2? or is this not something we’d generally want to have?
#2022-12-0219:33Brett RowberrySay I host an EQL endpoint. How do I give clients an OpenAPI/Swagger/GraphiQL experience?#2022-12-0220:08wilkerlucioyou can expose the indexes (eg: using boundary interface on pathom3), this is the raw data of the graph, tooling can be built around that, I provide Pathom Viz as a official UI for that purpose#2022-12-0220:19hadilsI have been trying to wrap my head around placeholder queries to no avail. Can someone post some examples/explanations? Thanks in advance!#2022-12-0614:13dehliplaceholder queries allow you to hydrate pathom input whicn you can then ask for keys underneath. For example, if you had a resolver "full name" that took first name and last name you could query for it like:
[{(:>/user-one {:user/first-name "Jane"
                :user/last-name "Doe"})
  [:user/full-name]}

 {(:>/user-two {:user/first-name "John"
                :user/last-name "Smith"})
  [:user/full-name]}]
It would then return data like:
{:>/user-one {:user/full-name "Jane Doe"}
 :>/user-two {:user/full-name "John Smith"}}
Let me know if that helps explain it for you! There's also https://pathom3.wsscode.com/docs/placeholders/ which goes in more depth
#2022-12-0614:48hadilsThank you @U2U78HT5G. So full-name is a parameterized query, from what I gather from your EQL?#2022-12-0614:49dehlifull name is a resolver that takes two inputs. something like:
(pco/defresolver full-name
  [{:user/keys [first-name last-name]}]
  {:user/full-name (str first-name last-name)})
#2022-12-0614:50dehliyou could instead do something like:
[{(:>/user {:user/id "foo" :user/first-name "John"})
  [:user/full-name]}]
and it would take the input :user/id and :user/first-name and figure out how it can get :user/last-name to fulfill the requirements of :user/full-name
#2022-12-0614:51dehliso if you had a resolver that took :user/id and spit out :user/last-name it would make that request to get the last name before passing the data to full name#2022-12-0614:52dehliplaceholders are more powerful than parameters b/c they can be used as inputs in any other resolver#2022-12-0616:52wilkerlucioto help understanding placeholdes, we can talk about the primary use case, that is to be able to split the view of some entity between multiple components in a tree#2022-12-0616:54wilkerluciofor instance, lets say you have a movie, and wanna break the representation in a few different components: • basic data • movie gallery • movie comments#2022-12-0616:55wilkerlucioand here the queries for each component: • basic: [:movie/id :movie/title :movie/director-name] • gallery: [{:movie/images [:image/url :image/thumbnail-url]}] • comments: [{:movie/comments [:comment/id :comment/message]}]#2022-12-0616:56wilkerlucionow, lets say you have a Fulcro component that uses the :movie/id as identity, and wanna use those components inside#2022-12-0616:56wilkerlucioyou can't just merge their queries, that would completly break Fulcro, but even without fulcro, merging queries has its own issues with conflicts (maybe different components what the same thing using different params for example)#2022-12-0616:57wilkerluciothat's where placeholders come, they allow a parent entity, to represent different subparts (the subqueryes) at different levels, but keeping the identity of the parent#2022-12-0616:58hadilsThank you @U066U8JQJ. I am not using Fulcro. I don’t know what a ā€œparentā€ is in this context.#2022-12-0616:58wilkerluciohere is the solution in this case, a wrapper component using:
[:movie/id
 {:>/movie-basics [:movie/id :movie/title :movie/director-name]}
 {:>/movie-gallery [{:movie/images [:image/url :image/thumbnail-url]}]}
 {:>/movie-comments [{:movie/comments [:comment/id :comment/message]}]}]
#2022-12-0616:58wilkerluciothis way, we have each different context (each sub-section requirement) separated in its own query part#2022-12-0616:59wilkerlucioso they have isolation, but still share context#2022-12-0617:00wilkerluciothe params part that @U2U78HT5G described is more a helper that got add in Pathom 3, and I may have to break that interface, because it prevents some custom usages of params with placeholders#2022-12-0423:28hadilsI want to insert meta-data into the result of a resolver or mutation. This meta-data is read by the process that sends the result back to the client. I am pretty sure that I can do it with plugins, but I don’t know where to put the meta-data. Any suggestions? Thanks in advance!#2022-12-0516:20sheluchinSay I have an established path for getting :x. Now my application is evolving, and I have a new way of getting :x. How should I evolve my application? Options seem to be: 1. replace the code getting :x using the old way 2. add new resolvers which also get :x, but give them a higher priority 3. give :x a different name when it's retrieved using the new way 4. create a new wrapper route, like [{:new-path [:x]}] 5. change the old resolver by adding a new-way? param 6. something else?#2022-12-0522:12dehlidoes the new way of getting :x take different input?#2022-12-0522:46sheluchin@U2U78HT5G nope#2022-12-0522:51dehliThen i would just remove the old resolver. It should be transparent to clients if input and output are the same.#2022-12-0523:05sheluchinWhat if the old resolver may still be useful in some limited circumstances in the future and I don't really want to completely lose it?#2022-12-0523:07dehliWhat makes it useful over the new resolver?#2022-12-0523:11sheluchinI kind of want to keep the question abstract. But let's say they are both ways to calculate a product, but the new way is more efficient. Maybe I want to use the old one to be able to compare old VS. new performance.#2022-12-0523:15dehliIn that case I would give it a new key where it's clear that it's been replaced so you don't accidentally pull it in unless you really want to for performance checks, etc.#2022-12-0523:16sheluchinThat makes sense but I've seen it repeated many times that you should never change the meaning of a name.#2022-12-0614:03wilkerluciohello @UPWHQK562, its right that we should aim to keep the semantics of the name consistent, but you can change the implementation as much as you want, as long as the name keep its meaning, being able to change the impl is a design goal of all of this šŸ™‚#2022-12-0614:04wilkerlucioif you want to write test to compare old vs new impl, you can have one env with the old, and one with the new, this run you can run the same queries and compare the results, given everything else other than this resolver is being replaced#2022-12-0614:42sheluchinThanks @U066U8JQJ and @U2U78HT5G!#2022-12-0616:25pithylessHow I understood your criteria: From the outside clients perspective, they want :x (don't care about implementation). From the inner developer perspective, you may want to differentiate performance/implementation/etc.. but "not always". I would actually move the original implementation into a new key :x-1 and the new implementation into :x-2. Then have :x resolver dispatch to :x-1 or :x-2 (maybe always to the new one, maybe via feature flags, or maybe totally dynamic 🤷) Client always sees :x and the internal systems may ask for generic :x or more directly for :x-1 or :x-2. Probably would be good to choose names and namespaces so that you know that the latter are specific implementations of the former, but if it's important to your domain (and your internal systems/developers are just more "clients" of your API), why not just use the existing EQL machinery to name and identify that this is an important distinction for you?#2022-12-0911:23sheluchinThanks for the thoughtful take, @U05476190. That does indeed cover it well. Considering that the developer may have a different sense of "meaning" and that the model should accommodate for it in some way adds an interesting layer. From that perspective the implementation is part of the meaning of the thing. And why shouldn't the developer be included as a class of user? The developer probably uses the thing more than anyone šŸ™‚ I love that there are so many ways to interpret and represent data requirements through attribute modelling. There's so much art to it.#2022-12-0623:53Tommyis there a way to update the resolver env in between running a mutation, and resolving the queries it is joined on? my use case is that I put an immutable DB view at the start of each pathom request, but after a mutation, i need to update that view to a newer one#2022-12-0700:35TommyI am going to use ::pcr/wrap-mutate plugin, will update if it works#2022-12-0702:47Tommythat did not work, I don't see a way to modify the env for the mutation's query from the wrap-mutate function#2022-12-0702:52pithylessI’ve solved this problem in pathom2 and pathom3 by passing in the DB as an immutable value wrapped in an atom (and possibly a separate DB conn if necessary for mutations) in the env. The queries should deref the db-value atom and the mutations should swap! the db-value atom after successful write with the updated DB.#2022-12-0702:54pithylessI understand there was some separate mechanism for updating env in p2 by returning a special key, but IIRC this was removed in p3. Not sure if there is a more idiomatic way to do this in p3 now, but this approach has worked for me :)#2022-12-0702:55Tommyah that is clever, thank you#2022-12-0721:11TommyI found a problem and filed an issue: https://github.com/wilkerlucio/pathom3/issues/174 . The pbip/mutation-resolve-params`` plugin is broken because of this issue. Will look into fix#2022-12-0721:33Tommysent in a PR with quick fix https://github.com/wilkerlucio/pathom3/pull/175#2022-12-0912:57wilkerluciothanks Tommy! I remember going over this at some point, will have a look at your PR. I'm traveling this weekend out of computer, so I'll check it next week.#2022-12-1821:11JHi guys! I have an error that I don’t understand. I reinstall multiple time my .m2 but I already have this error. Have you experienced this kind of things? I try to require the namespace com.wsscode.pathom3.connect.indexes#2022-12-1821:33JSolved! It needed Java11#2022-12-2310:14mitchelkuijpersI am running into some performance issue with pathom3 when I return a list of around 2002 items#2022-12-2310:14mitchelkuijpers#2022-12-2310:14mitchelkuijpersit's a resolver which returns everything that is needed in one list but for some reason it seems to trigger 2002 extra resolver calls#2022-12-2312:13mitchelkuijperspco/final-value was my solution, this stops the postprocessing of results#2022-12-2616:10Jakub Holý (HolyJak)Hi! In Pathom 2, how can I make a derived value that depends on 1 - 2 other values? Namely I want to make
(defresolvers dimensions [env {:product/keys [dimension-a dimension-b]}]
  {::pc/input #{:product/dimension-a :product/dimension-b}
   ::pc/output [:product/dimensions]}
  {:product/dimensions (if (and dimension-a dimension-b)
                                          (str dimension-a " x " dimension-b)
                                          (str "A=" dimension-a))})
where dimension-a is always defined but dimension-b is optional. šŸ™
#2022-12-2616:52souenzzowe don't have optional inputs in pathom2 options: • make dim-a nillable • make only b as input, then call (parser env :a) inside the resolver.#2022-12-2915:57hadilsHi! In Pathom 3, what is the difference between the :dispatch_key and the :key in the AST?#2022-12-2916:00wilkerluciohello, pathom 3 still relies on the same eql library, which is the one that does the parsing for eql to ast. the reason they exist is historical (from om.next), their difference happens when the key is an ident, for eg, the key [:user/id 123] will have the dispatch key :user/id. Pathom 3 doesnt use dispatch key for anything#2022-12-2916:02hadilsThank you @U066U8JQJ!#2022-12-2918:18hadilsHi! I am writing a plugin in Pathom 3. I need to capture the result of a mutation. What is the best way to accomplish this?#2022-12-2923:13wilkerlucioyou can use the hook point wrap-mutation: https://pathom3.wsscode.com/docs/plugins#pcrwrap-mutate#2022-12-2923:15hadilsI am using wrap-mutation. How do I capture the result of the mutation within this plugin? Can you show me an example?#2022-12-2923:18wilkerluciosure, one moment#2022-12-2923:19wilkerlucio
(p.plugin/defplugin capture-mutation
  {:com.wsscode.pathom3.connect.runner/wrap-mutate
   (fn [mutate]
     (fn [env params]
       (let [response (mutate env params)]
         (println "Captured response -> " response)
         response)))})
#2022-12-2923:20wilkerlucionote that if are you using the async processor the response might be a channel (in case the mutation returns an async response)#2022-12-2923:24hadilsI have tried this. I also accounted for async responses. However, the shape of result is different than what is output from a mutation, namely the mutation name is missing. I have added that with code. Is that the proper way to do it?#2022-12-2923:25wilkerlucionot sure what you mean the shape of result is differnet, that mutate fn should be the exact one that your mutation runs on, the return of it becomes the return#2022-12-2923:26hadilsNot exactly. The response would be something like:
{:result "foo"}
but the result of executing the mutation is:
{'echo/bar {:result "foo"}}
#2022-12-2923:27wilkerluciothe mutation will not return its name on it, Pathom adds that later to the final response, but the user response isn't supposed to have that key in it#2022-12-2923:27hadilsAh.#2022-12-2923:27wilkerlucionot to confuse the mutation response (map out of the mutation call) with the processor response, which is the final part#2022-12-2923:28hadilsThank you @U066U8JQJ. I will write my subscribers to not take the mutation name.#2022-12-2923:28wilkerluciono worries, but in the process of looking it up now, I realize there is no easy way to know at that point what mutation is being invoked#2022-12-2923:29hadilsYes true.#2022-12-2923:29wilkerluciothis is something easy to fix, I can add it to the env before running it, this way in hte plugin you can tell which mutation is that#2022-12-2923:29hadilsI have found it in the ast as the :key member#2022-12-2923:30hadilsI just wanted to confirm that adding it is correct, or if i was missing something.#2022-12-2923:30wilkerluciothe problem with the one there is that there might be multiple of them#2022-12-2923:30wilkerlucioif you have multiple mutations in the same process call, you wouldn't be able to tell which key is being executed#2022-12-2923:31hadilsI didn’t think of that. How do you propose to add it to env#2022-12-2923:31wilkerlucioI'm gonna add to Pathom, so it will be there, still have to pick a key for it#2022-12-2923:31wilkerlucioin the case of resolvers there is one already, the ::pcp/node, which has the node information for that resolver call
#2022-12-2923:32hadilsOk, thanks @U066U8JQJ! I am grateful to you for your help.#2022-12-3005:15caleb.macdonaldblackHow is everyone finding the smart-map in Pathom3? I find myself reaching for p.eql/process mostly. The smart-map just seems kinda gimmicky. Would love to hear from people that use it more.#2023-01-0212:15wilkerluciohello Caleb, I personally haven't used it much either#2023-01-0212:16wilkerlucioI can imagine some interesting usages when interfacing with things that require maps#2023-01-0212:17wilkerlucioone idea I have for instance, is for a compatibility layer between Pathom 2 and 3, where a Pathom 2 plugin for instance can receive an env that's a smart map, that can respond to Pathom 2 env keys, translating to Pathom 3 keys#2023-01-0214:16caleb.macdonaldblackAh that’s interesting. For smaller things it could be useful. One project I’m working on isn’t using Pathom at all. We have a data structure that we’ve began refactoring by renaming some keys with improved namespaces & names. Also small structure changes. All new keywords however to avoid breaking changes. What we will essentially have is the old and new data structures merged together. I could wrap the data structure in a smart map with resolvers providing bidirectional conversion for old keywords to new keywords. It’s a very cool solution to this compatibility problem. However, there may be situations where transformative code might take the smart map and return a regular map. Also when sending smart maps through legacy code, developers reading the code won’t have any idea they’re dealing a smart map unless they know before hand. #2023-01-0214:23wilkerlucioyup, all you said is correct, I add one caveat to the "However, there may be situations where transformative code might take the smart map and return a regular map.", this is a config on smart maps, by default, any map (or sequence of maps, including sets) will get automatically wrapped to become a smart map too (check: https://github.com/wilkerlucio/pathom3/blob/98098d2827fa3f14aab6ee5e7c265cb3db4c94bb/src/main/com/wsscode/pathom3/interface/smart_map.cljc#L79-L101)#2023-01-0214:23wilkerlucioyou can set ::psm/wrap-nested? false (in env) to disable this behavior#2023-01-0214:29caleb.macdonaldblackThanks!#2023-01-1317:19Eric DvorsakI haven't been using them at all#2023-01-0208:20robert-stuttafordhey @wilkerlucio! so we're fiiiiinally getting to spiking things with P3 this month, with a view to using it for major parts of our app logic. one of the big stumbles i had when researching with P3 before was working with Datomic dbs and using pathom-viz (which tried to read the whole datomic db to visualise it). i vaguely recall there being a way to hack it. is there perhaps more formal support for the viz connector now, to provide some control over how certain values are sent? basically i'd like my Datomic db to print as it does in the repl {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "2d494c594240444e03494f03694f6d4c1b4b1a1d141848"}, :content ("[emailĀ protected]")}, rather than all its internals šŸ™‚#2023-01-0208:22robert-stuttafordcc @U0HJ7CX6H#2023-01-0212:14wilkerluciohello Robert, happy new year šŸ™‚#2023-01-0212:15wilkerluciofor datomic, in general I expect the DB value to remain on env, which shouldn't get exposed to Pathom Viz, can you tell more about how/why it gets out there?#2023-01-0213:37robert-stuttafordhappy new year @wilkerlucio! we'll take a fresh stab and see if it still gets in the way!#2023-01-0217:16sheluchinAre many people using Pathom in data processing pipelines?#2023-01-0217:39caleb.macdonaldblackSort of. I’ve been working with renewable energy data and using Pathom3 to process and run simulations.#2023-01-0217:40caleb.macdonaldblackThe building blocks of fetching, pre-processing, simulation and aggregating are using Pathom3. While all this is coordinated with pathom. The processing of individual collections of data are done within a single pathom resolver.#2023-01-0418:11Pragyan TripathiWe have built dynamic pathom resolvers over Apache Spark and use that for data processing purposes.#2023-01-1015:40jjttjjI'm faced with using a big java library with tons of interconnected types that are hard just to construct, at least before I'm past the learning curve. I'm tempted to just do some sort of analysis on the java and use pathom as a construction engine, filling in default values in some places. Anyone use pathom like this?#2023-01-1016:29souenzzoI tried to do something like that with AWS SDK long time ago. Not simple, but seems to be possible#2023-01-1016:40souenzzo(I gave up because it was out of scope of my current task)#2023-01-1017:03jjttjjYeah I've found that to be a problem I hit with pathom. A problem starts to look graphy and seems like it would be great to work with it via pathom. But when I start to code up the resolvers it becomes apparent that there are a lot of decisions to make and that the wrapper itself would be a whole project that may not be justified. This isn't at all a critique of pathom just a pattern I catch myself in where I want to use it too frequently or possibly for the wrong things. Which I think is why I wanted to post the idea here first#2023-01-1215:09Ben Grabow@U064UGEUQ I've stumbled across https://github.com/nivekuil/nexus that is a dependency injection library using Pathom under the hood. I haven't used it myself, but it seems to fit in the direction you're proposing.#2023-01-1511:50robert-stuttaford@wilkerlucio i highly recommend adding a big notice to https://github.com/wilkerlucio/pathom readme to say that https://github.com/wilkerlucio/pathom3 exists and you should go there if you're starting with pathom šŸ™‚ also, same thing but on the older doc site#2023-01-1813:36wilkerluciothanks for the pointer, notes add: https://github.com/wilkerlucio/pathom#pathom-3 https://blog.wsscode.com/pathom/v2/pathom/2.2.0/introduction.html#_pathom_3#2023-01-1813:55robert-stuttafordoh fab thank you @wilkerlucio šŸ‘#2023-01-1517:41sheluchinWhat is the right way to think about pbir/equivalence-resolver? If it creates a bi-directional transformation, doesn't that kind of imply that it's creating a cycle in the DAG?#2023-01-1519:07wilkerluciohello, good question, indeed they do create a cycle, but thats not really an issue, because pathom planner can detect the cycle and skip it, the idea of a bidirectional link is that in case you have one of the sides available, it can translate it into the other#2023-01-1519:08wilkerlucioand to start from it, it could be some provided information, or getting resolved via some other resolver#2023-01-1519:08wilkerluciomakes sense?#2023-01-1519:09sheluchinHi @wilkerlucio. If I understand correctly, preventing cycles is important when creating and executing the plan, but it's fine to have cycles in the initial available graph?#2023-01-1519:11wilkerlucioā€œpreventing cycles is importantā€, not sure what you mean there#2023-01-1519:12sheluchinPathom has cycle detection, but is it also important to avoid creating cycles when writing resolvers?#2023-01-1519:20wilkerluciocycles will happen naturally, its not something you should require special attention#2023-01-1519:21wilkerlucioreducing the number of paths for each attribute is encouraged since its gonna help keep paths clean, making planning / running faster#2023-01-1519:23sheluchinAh, so it's perfectly fine to have an a->b resolver and a b->a resolver.#2023-01-1519:23wilkerlucioyup, in fact that is exactly what the equivalence resolver is, it makes two alias resolvers, one in each direction#2023-01-1519:25sheluchinOkay, that simplifies my thinking. I thought it's necessary to pay special attention to make sure that your resolver graph remains a DAG - as in, being careful that you don't define cycles.#2023-01-1519:25wilkerluciono need, the planner will make sure cycles dont happen#2023-01-1519:26wilkerlucioand the graph generated by the planner will always be a DAG#2023-01-1519:26sheluchintyvm @wilkerlucio, that clears it up for me.#2023-01-1813:55robert-stuttafordoh fab thank you @wilkerlucio šŸ‘#2023-01-1913:46Jakub Holý (HolyJak)Hello! A question about P2. Fulcro generates this query: [:order/id :order/product-count :order/total-cost {:order/product [:product/name]}] (an order has exactly 1 product). Now the problem is that :order/total-cost is a derived value, simply said it is :order/product-count * :product/price . How can I write a resolve that produces this total-cost? I can make one that takes order/id, fetches the product price from the db and computes but that will result into 2 db queries for the same product - this one for price and the EQL-triggered one for the name. Is there some smart trick I can use so that only one db query will be needed? šŸ™#2023-01-1913:49wilkerlucioin case :produce/price is inside :order/product (which if I understand correctly is the case), them there is not a really clean way to do that in Pathom 2 a "not so clean" way is to make a sub-request from inside the :order/total-cost resolver, something like (parser env [{:order/product [:order/price]}], them you extract the price and multiply by the count (which can be a regular input for the resolver)#2023-01-1913:51wilkerlucioin Pathom 3, you can have nested requires, which allows you to "fix" this doing something like:
(pco/defresolver order-total [{:order/keys [product-count product]}]
  {::pco/input
   [:order/product-count
    {:order/product [:product/price]}]}
  {:order/total-cost (* product-count (:product/price product))})
#2023-01-1913:52wilkerlucioand talking about "order modeling", a common pattern is to also have :order/line-items, and the count goes there (as each product), otherwise your order can have only one type of product (which might be your case, if so, them your current design works fine)#2023-01-1913:54Jakub Holý (HolyJak)Thanks a lot, Wilker! Your help is most appreciated. I guess I should finally bite the bullet and upgrade to P3.#2023-01-1913:55wilkerluciodepending on how your :order/product is implemented, you could also just ask it flat in the resolver, as long as you are sure the price will be included in it#2023-01-1913:55wilkerluciolike:
(pco/defresolver order-total [{:order/keys [product-count product]}]
  {::pco/input
   [:order/product-count
    :order/product]}
  {:order/total-cost (* product-count (:product/price product))})
#2023-01-1913:56wilkerlucioit has the downside of being a transitive dep (meaning you are responsible for making sure :product/price comes in :order/product), but works#2023-01-1913:59Jakub Holý (HolyJak)What do you mean by > for making sure :product/price comes in :order/product ? The incoming query (shown in the original message) only asks for order/product -> product/name and the resolver used for that, product-id-resolver, only returns the properties that have been asked for it, i.e. it will return only {:product/id ..., :product/name ...} . Do you mean that if it instead eagerly always returned the product/price then I could write the resolver as demonstrated above?#2023-01-1914:00wilkerlucioI mean the resolver that implements the output of :order/product#2023-01-1914:00wilkerluciothe query on the client shouldn't need any change#2023-01-1914:01Jakub Holý (HolyJak)The resolver returning :order/product is the order-id-resolver and it only returns {... :order/product {:product/id "xxx"}} , all the product details are fetched via the product-od-resolver subsequently#2023-01-1914:01wilkerluciook, yeah, in this case you can't do the flat thing I was talking about#2023-01-1914:01Jakub Holý (HolyJak)(These are RAD auto-generated resolvers, though I could also write a custom one just for this)#2023-01-1922:34hadilsI need a sanity check. I would like to return {ident1 {map of values} ident2 {map of other values}…} from a mutation (this is for subscriptions to be able to pick off the data by ident). Can I do this, and if so, would i specify the output thus:
{::pco/output [{:ident-selector [:value1 :value2 :value3]}]
where an ident is [:ident-selector ident-value] and the vector following it is the return map?
#2023-01-2020:07wilkerluciohello, that's an interesting question, because it touches something very less known about pathom. the issue when dealing with structures like this is that there are 2 different semantics for maps: 1. maps as an entity, in which each key is some attribute of that entity, this is the default behavior 2. "map containers", in those cases, the map is not an entity, but a collection of entities, where the key is some index of an entity, but not an entity attribute#2023-01-2020:08wilkerluciothere is feature to handle this, there is marking a map as map container (by adding the ::pcr/map-container? meta at the map)#2023-01-2020:08wilkerlucioI was trying to give you here a full example, but as I was trying it, it isn't working as I expected, its likely a bug on that part that I need to work on#2023-01-2020:09wilkerluciocan you please open an issue to track this problem?#2023-01-2020:09hadilsSure. Thank you @U066U8JQJ#2023-01-2020:11hadilsDone.#2023-01-2020:15wilkerluciothank you!#2023-01-2622:23wilkerlucio@UGNMGFJG3 just merged the fix, you should be able to use map-container properly using the main branch, please let me know if you still find any issue#2023-01-2622:25hadilsThanks @U066U8JQJ !#2023-01-1923:52Jakub Holý (HolyJak)~Hello again! In P3, how to troubleshoot~ > ~ā€œBatch results must be a sequence and have the same length as the inputs.ā€~ ~? I.e. why does Pathom think this is a batch result/resolver?~ I did have batch? true šŸ˜…#2023-01-2000:02Jakub Holý (HolyJak)~Q1: Why might I be getting ~ The problem seems to have been with a resolver returning LazySeq as the value of one attribute; turning map to mapv fixed it. ~Q2: Why am I getting ~ Reading the source of com.wsscode.misc.coll/restore-order it seems that indeed returning #2023-01-2000:20Andy Carlileim a noob to pathom, om, and graphql. i'm losing my noodle over querying by id: https://clojurians.slack.com/archives/C053AK3F9/p1674170178447079#2023-01-2001:01Jakub Holý (HolyJak)Another P3 mystery with a join. This returns as expected:
(parser {} [{[::person/id "ann"] [::person/addresses]} ])
; =>
{[::person/id "ann"] {::person/addresses [{::address/street "First St."}
                                          {::address/street "Second St."}]}}
but if I change it to specify I only want .../street for each address:
(parser {} [{[::person/id "ann"] [{::person/addresses [::address/street]}]} ])
; =>
{[::person/id "ann"] #::person{:addresses [{} {}]}}
no data is in the addresses 🤯 There is a single person-id resolver involved and it returns all the data needed, like this:
[#::person{:id "ann", :addresses [#::address{:id "a-one", :street "First St."} #::address{:id "a-two", :street "Second St."}]}]
#2023-01-2002:46wilkerluciohello, that's seems off, but maybe its something with idents, can you make a repro please?#2023-01-2121:43Jakub Holý (HolyJak)Hi Wilker! I found out that of course the problem was with my resolver, I just did not see it šŸ˜…#2023-01-2200:47Jakub Holý (HolyJak)@U066U8JQJ I have figured out 4 ways to fix my code ā˜ļø to return the expected data but still do not understand why the version I have is ā€œbrokenā€ and returns [{} {}] instead of the addresses. The thing is that I have 2 resolvers: 1) a person resolver that returns the whole data, 2) an address resolver for :address/id -> :address/street. The 4 ways to fix the resolution to return the expected street names are: 1. Change the person/id resolver’s outputs to declare [ .. :person/addresses ..] instead of [.. #:person{:addresses [:address/id]} ..] 2. Change the person/id resolver’s outputs to also include address/street, i.e. [.. #:person{:addresses [:address/id :address/street]} ..] 3. Remove the resolver for address/id → address/street 4. Change the person/id resolver to only return address/id and not the street Here is the simplest possible code to demonstrate the issue (in its ā€œbrokenā€ state):
(def asami-p3-pers-resolver-diy
  (pco/resolver
    {::pco/op-name 'asami-p3-pers-resolver-diy
     ::pco/batch?  true
     ::pco/input   [:person/id]
     ::pco/output  [; fix 1: use just `:person/addresses` inst. of the join below
                    #:person{:addresses [:address/id ; fix 2: add `:address/street` here
     ]}]
     ::pco/resolve (fn [_env in]
                     (println "JHDBG: asami-p3-pers-resolver-diy" in)
                     [#:person{:id "ann", :addresses [#:address{:id "a-one", :street "First St."}
                                                      #:address{:id "a-two", :street "Second St."}]}])}))

(def asami-p3-addr-resolver-diy
  (pco/resolver
    {::pco/op-name 'asami-p3-addr-resolver-diy
     ::pco/batch?  true
     ::pco/input   [:address/id]
     ::pco/output  [:address/street]
     ::pco/resolve (fn [_env in]
                     (println "JHDBG: asami-p3-addr-resolver-diy" in) ; never called!
                     [#:address{:id "a-one", :street "First St."}
                      #:address{:id "a-two", :street "Second St."}])}))

(p.eql/process
  (pci/register [asami-p3-pers-resolver-diy 
                 ; fix 3: comment out the resolver below:
                 asami-p3-addr-resolver-diy])
  [{[:person/id "ann"] [{:person/addresses [:address/street]}]}])
What I do not understand why the presence of the address resolver makes the data disappear, even though the resolver is never called. Thank you very much for any insights! PS: My whole laborious exploration is recorded in https://gist.github.com/holyjak/9951076cbaaac945be43cec98e2e41b0#troubleshooting-auto-generated-fulcro-rad-id-resolvers-in-pathom-3
#2023-01-2515:14Jakub Holý (HolyJak)@U066U8JQJ sorry to bother you but do you think ā˜ļø is a bug and should I create an issue for it?#2023-01-2613:14wilkerluciohello @U0522TWDA, sorry the delay, I plan to have a proper read and get back to you later today#2023-01-2613:18Jakub Holý (HolyJak)You don’t need to read the whole thread, just the last message with code šŸ™‚ thank you!#2023-01-2621:31wilkerluciohello, just took the time to run your code, I have one more fix šŸ˜› there is, remove ::pco/batch? from asami-p3-pers-resolver-diy, which makes me think its same issue reported in #173 and #177, also this discussion here: https://clojurians.slack.com/archives/C87NB2CFN/p1674636918359429#2023-01-2621:33wilkerlucioand thanks for the extra repro, good that I will by able to validate the solution in a few different examples#2023-01-2621:37wilkerlucioaltough, for your case, the fix 1 is the correct way to do it, since the street is available at the output data#2023-01-2621:38wilkerluciobut it still a Pathom issue on how its handling it without, it should be able to figure out (although less optimal, given it could avoid some things if it knew the exact nested shape)#2023-01-2621:42wilkerluciojust found something that might make your case different#2023-01-2621:44wilkerlucio
(p.eql/process
    (pci/register [asami-p3-pers-resolver-diy
                   asami-p3-addr-resolver-diy])
    {:person/id "ann"}
    [{:person/addresses [:address/street]}])
#2023-01-2621:44wilkerluciothis works fine ā˜ļø#2023-01-2621:44wilkerlucioso there might be something with batch + idents that's off here#2023-01-2621:47wilkerlucio@U3XCG2GBZ this observation also matches your report, this also works in your setup:
(p.eql/process env
    {:child/id 1}
    [{:child/favorite-toy [:toy/name]}])
#2023-01-2709:52Jakub Holý (HolyJak)The problem here is that these resolvers are auto-generated. And they may sometimes return more nested data, sometimes only the (declared) id. So changing the :outputs is not really an option. I have a workaround for now, just need to make sure the resolvers do not return anything extra. But it would be nice to be able to do that. FYI The problem popped-up in in https://github.com/holyjak/fulcro-rad-asami/blob/18729aaea22b37c2cbf2d00a08a62f87b1b37030/src/cz/holyjak/rad/database_adapters/asami/pathom_common.cljc#L110 plugin. It gets triggered when people insert data manually and incorrectly (i.e. sub-entities such as addresses as nested entities instead of top-level entities). Which should normally not happen if they use the plugin as designed. Thank you for looking into this!#2023-01-2710:34wilkerluciountil there is a proper fix, one solution is to use the parallel runner, the parallel runner uses a different mechanism for batching and doesn't have this bug#2023-01-2710:35wilkerluciofrom your example, this works fine:
@(p.a.eql/process
     (pci/register
       {::p.a.eql/parallel? true}
       [asami-p3-pers-resolver-diy
        ; fix 3: comment out the resolver below:
        asami-p3-addr-resolver-diy])
     [{[:person/id "ann"] [{:person/addresses [:address/street]}]}])
#2023-01-2711:33Jakub Holý (HolyJak)Good to know, thank you!#2023-01-2004:27PanelWould anyone have an example of using dynamic resolvers to batch requests to an external api ?#2023-01-2013:42wilkerluciohello Panel, can be a bit more specific? what kind of external api? is it a simple HTTP request? I'm wondering if you really want a dynamic resolver, or just a batch resolver#2023-01-2022:02PanelI’m wrapping a xml-rpc api. You can describe the data you want with xml and get many different entities back in a single http call. So in pathom I could have a resolver per entity that make a http call each but I could also find a way to batch those http call because the xml-rpc endpoint allow to send many request at once.#2023-01-2116:18wilkerluciogotcha, in general the process here is to figure a out to translate the foreign schema in Pathom resolvers, can you instropect that API to get its description?#2023-01-2121:48PanelYes I know what entities I can fetch and the payload I need to get them. I made a resolver per entity and it works but it’s making a lot of extra request by not using the ability to fetch multiple entities per call. It’s like if I had a rest api where I could send it [/user/1/adresse /user/1/pets /user/1/keys] Instead of making 3 separate call. If dynamic resolvers is the way to go, I think need to figure out how to define the ast for my foreign api.#2023-01-2213:57wilkerluciothe ast gets build by pathom for you to process via the dynamic resolver, in case of batches its the same, but you get multiple things, and have to figure how to send out/process back of them in one request#2023-01-2323:25wilkerlucioif that's something you can share, I could look at how to make it with you#2023-01-2022:04wilkerlucio#2023-01-2115:23caleb.macdonaldblackGreat work!#2023-01-2211:22paramemeIn the last non command-line syntax block in the tutorial code above the concept or symbol woeid is only referenced twice in the registration in the env and doesn’t seem bound / defined. Am I missing something?#2023-01-2213:55wilkerlucioups, thanks for the pointer, removed the last reference of woeid that I found from the previous tutorial: https://github.com/wilkerlucio/pathom3-docs/commit/3dedbf24fcf4c0b5ed24ca5ab8f33df222a8a733#2023-01-2116:17sheluchinI'm doing lots of work in Python lately. Has anyone been able to leverage Pathom outside of Clojure projects to gain some advantage?#2023-01-2122:43caleb.macdonaldblackI have some thought about the semantics of nested entities I’d like to share. Pathom3 treats the attributes pointing to nested entities as a value whereas a relationship might make more sense. If the behaviour was a little different then it could allow for some useful queries (Assuming it would work). I’m aware that Pathom3 is not designed to support a query as described in the snippet. The recommended approach being to ā€œPass data downā€ to allow a child to access an attribute from its parent. The reason Pathom3 doesn’t can’t do this query is because family-member-surnames has :com.example.family/members as both it’s input and output. However semantically, the family-member-surnames isn’t actually transforming the :com.example.family/members attribute at all. The attribute represents a relationship between the family and person entities and the resolver isn’t changing anything about that relationship at all. The same entities are returned, in the same order, and with no values changed, only added. So the value of the com.example.family/members attribute isn’t really children and their attributes, but instead a reference to the children. So it would be reasonable to expect Pathom to treat attributes with nested data as relationships instead. It could work by generating and caching a reference to all nodes/entities in nested data while running. Planning would detect nested input/output and dynamically create reverse lookup resolvers that will output a reference to a parent entity in exchange for child reference. And the parent reference could then be exchanged for parent attributes. So each map within a deeply nested data-structure is assigned a reference by Pathom3, and then cached. And that reference can be used reach parent data. Additionally, planning could detect and merge nested data across resolvers if the attribute is the same, which indicates it’s the exact same relationship. Same for a resolver that has the same nested attribute in the input and output. Here’s an example of some code that doesn’t work but theoretically could.
(ns com.example.core16
  (:require
    [clojure.test :refer :all]
    [com.wsscode.pathom3.connect.built-in.resolvers :as pbir]
    [com.wsscode.pathom3.connect.indexes :as pci]
    [com.wsscode.pathom3.connect.operation :as pco]
    [com.wsscode.pathom3.interface.eql :as p.eql]))

(def family-db
  (pbir/constantly-resolver
    :com.example.family/db
    {"Family1" {:com.example.family/surname "Smith"
                :com.example.family/members [{:com.example.person/first-name "John"}
                                             {:com.example.person/first-name "Mary"}]}}))

(def family-by-id
  (pbir/attribute-table-resolver
    :com.example.family/db
    :com.example.family/id
    [:com.example.family/surname
     {:com.example.family/members [:com.example.person/first-name]}]))

(pco/defresolver family-member-surnames
  [_env {:com.example.family/keys [members surname]}]
  {
   ::pco/input  [:com.example.family/surname {:com.example.family/members [:com.example.person/first-name]}]
   ::pco/output [{:com.example.family/members [:com.example.person/surname]}]}
  {:com.example.family/members
   (mapv (constantly {:com.example.person/surname surname}) members)})

(def env
  (pci/register
    [family-db
     family-by-id
     family-member-surnames]))

(deftest derive-person-surname-from-family-test
  ;; Fails
  (is (= {:com.example.family/members
          [{:com.example.person/first-name "John"
            :com.example.person/surname "Smith"}
           {:com.example.person/first-name "Mary"
            :com.example.person/surname "Smith"}]}
         (p.eql/process
           env
           {:com.example.family/id "Family1"}
           [{:com.example.family/members
             [:com.example.person/first-name
              :com.example.person/surname]}]))))
#2023-01-2122:46caleb.macdonaldblackI’ve also played around with Datascript and Pathom3 to apply this idea. It seems like it could work. However if possible, it would be preferable for the caching and references to be implicitly handled within Pathom3 as apposed to the explicit approach in the example. Here’s an example:
(ns com.example.core18
  (:require
    [clojure.test :refer :all]
    [com.wsscode.misc.coll :as coll]
    [com.wsscode.pathom3.connect.indexes :as pci]
    [com.wsscode.pathom3.connect.operation :as pco]
    [com.wsscode.pathom3.format.eql :as pf.eql]
    [com.wsscode.pathom3.interface.eql :as p.eql]
    [datascript.core :as d]))

(pco/defresolver family-members
  [_ {family :db/id :keys [conn]}]
  {::pco/output [{:com.example.family/members [:db/id :conn]}]}
  {:com.example.family/members
   (->> (d/pull @conn [:com.example.family/members] family)
        :com.example.family/members
        (mapv (fn [{:db/keys [id]}]
                {:db/id id
                 :conn  conn})))})

(pco/defresolver person-first-name
  [_ {person :db/id :keys [conn]}]
  {::pco/input  [:db/id :conn]
   ::pco/output [:com.example.person/first-name]}
  (d/pull @conn [:com.example.person/first-name] person))

(pco/defresolver person-full-name
  [_ input]
  {::pco/input  [:com.example.person/first-name
                 {:com.example.person/family [:com.example.family/surname]}]
   ::pco/output [:com.example.person/full-name]}
  (let [{:com.example.person/keys             [first-name]
         {:com.example.family/keys [surname]} :com.example.person/family} input]
    {:com.example.person/full-name
     (str first-name " " surname)}))

(pco/defresolver person-family
  [_ {person :db/id :keys [conn]}]
  {::pco/input  [:db/id :conn]
   ::pco/output [{:com.example.person/family [:db/id :conn]}]}
  {:com.example.person/family (-> (d/pull @conn [:com.example.family/_members] person)
                                  :com.example.family/_members
                                  first
                                  (assoc :conn conn))})

(pco/defresolver family-surname
  [_ {person :db/id :keys [conn]}]
  {::pco/input  [:db/id :conn]
   ::pco/output [:com.example.family/surname]}
  (d/pull @conn [:com.example.family/surname] person))

(def env
  (-> (pci/register [family-members person-first-name family-surname person-family person-full-name])
      (update ::pf.eql/map-select-include coll/sconj :db)))

(def entity
  (let [conn (d/create-conn {:com.example.family/members
                             {:db/cardinality :db.cardinality/many
                              :db/valueType   :db.type/ref}})
        {{:strs [smith-family]} :tempids}
        (d/transact! conn [{:db/id                      "smith-family"
                            :com.example.family/surname "Smith"
                            :com.example.family/members [{:com.example.person/first-name "John"}
                                                         {:com.example.person/first-name "Mary"}]}])]
    {:db/id smith-family
     :conn  conn}))

(p.eql/process
  env
  entity
  [:com.example.family/surname
   {:com.example.family/members [:com.example.person/first-name
                                 :com.example.person/full-name
                                 {:com.example.person/family
                                  [:com.example.family/surname]}]}])
#2023-01-2207:26jeroenvandijk(I think starting the thread with a summarizing question/remark and then details in the thread would make it easier to digest this:pray: )#2023-01-2300:01caleb.macdonaldblack@U0FT7SRLP Thanks for the feedback. I made some small changes I hope make’s it easier to read.#2023-01-2323:12wilkerluciohello @U3XCG2GBZ, I was looking at the first sources now, if I understand correctly, what you looking for is some way to merge the data that comes from sources under the same attribute. the problem with this is that it breaks an important invariant of how pathom processing works, there is: "once an attribute is set, its done, it can't change". I think that's what you mean when you say to threat as a relationship instead of a value. the problem is that, for pathom there is no such distinction, there is no concept of some sort of the value that could change after being set#2023-01-2323:13wilkerluciothe reason this invariant is important, is that it gives some sanity to the process, because if a value starts at something, is passed to some resolvers, and them at some point a new value emerges that changes it, it would affect the result of depending in the order of which the resolvers run, which would be caotic#2023-01-2323:14wilkerlucioit could be theoritically possible to try to observe all of that and make sure depences run in order to ensure the "full value" before sending it over, but its a really tricky problem as far as I see it#2023-01-2323:15wilkerluciothis is why you need to send the data down, because the children is processed after in the stack, so the parent isn't done, which means it can send things down, without mutating final values (by localizing the change down the stack)#2023-01-2323:16wilkerluciothis is also why we can't look up, fundamentally is because that parent value isn't ready, whcih means it could change after being looked up#2023-01-2323:16wilkerluciomakes sense?#2023-01-2322:00Ben GrabowIs there an easy way to ask Pathom "Here's an entity. Resolve all the attributes that are resolvable from that entity."? I have hacked together some naive code to do this using the io-index but I wonder if there is a robust, built-in way to do it.#2023-01-2322:01wilkerluciohello Ben, I suggest you look at https://cljdoc.org/d/com.wsscode/pathom3/2022.10.19-alpha/api/com.wsscode.pathom3.connect.indexes#reachable-paths#2023-01-2322:02wilkerluciothe reachable-paths will take an env and some entity, and will tell you all reachable paths#2023-01-2322:02wilkerluciothis is what Pathom Viz uses under the hood to provide completions#2023-01-2322:04Ben GrabowIt looks like I get the results in shape descriptor format, and I can pull the top-level attributes from the keys of this map. Is there a way to branch out to all nested attributes too? Or do I write the recursion myself?#2023-01-2322:05wilkerlucioI can't remember on the top of my head if that also does deep diving on nested, I guess no, but if that's case you can just repeat the process with the attributes at that level#2023-01-2322:09Ben GrabowHmm, I realize now the to-many resolver I have doesn't declare the nested attributes in the output spec, so I don't think I can make the jump automatically at planning time. I'll have to run the query first then iterate over the results to query into the nested attrs. reachable-paths does a ton of the heavy lifting for me though. Thanks!#2023-01-2322:27Ben GrabowStill pretty hacky, but this "works" for simple cases:
(defn recursive-resolve
  [env entity]
  (let [entity (p.eql/process
                 env
                 entity
                 (vec (keys (pci/reachable-paths env entity))))]
    (->> (for [[k v] entity]
           [k (cond
                (instance? PersistentArrayMap v) (recursive-resolve env v)
                (vector? v) (map #(recursive-resolve env %) v)
                :else v)])
         (into {}))))
At first instead of (instance? PersistentArrayMap v) I tried (map? v) but ran into trouble because my data includes some clojure.data.xml.node.Element objects which "are" IPersistentMaps but don't work as Pathom entities (nor should they).
#2023-01-2322:27Ben GrabowThis approach does not forward parent attributes to the child query context, although that may be useful for some cases.#2023-01-2322:28wilkerlucioI suggest you replace (vec (keys ... with (pfsd/shape-descriptor->query ...#2023-01-2322:29wilkerluciohttps://cljdoc.org/d/com.wsscode/pathom3/2022.10.19-alpha/api/com.wsscode.pathom3.format.shape-descriptor#shape-descriptor-%3Equery#2023-01-2322:31Ben GrabowThis doesn't seem to work smoothly since my input entity has some data in it:
(pfsd/shape-descriptor->query {:my-provided-entity {:body "hello world"}
                               :derived-from-my-entity {}})
Execution error (UnsupportedOperationException) at com.wsscode.pathom3.format.shape-descriptor/shape-descriptor->query$fn (shape_descriptor.cljc:138).
nth not supported on this type: Character
#2023-01-2322:32wilkerluciomaybe you removed the reachable-paths by accident? since that should return a valid shape descriptor#2023-01-2322:32wilkerlucioas: (pfsd/shape-descriptor->query (pci/reachable-paths env entity))#2023-01-2322:34Ben GrabowEliding the real data, my situation looks something like this:
(pci/reachable-paths env {:my-provided-attr {:body "hello world"}})
=>
{:my-provided-attr {:body "hello world"}
 :derived-from-my-attr {}}
The {:body "hello world"} is treated as a black box from Pathom's perspective. It should be an opaque piece of data, and :body is not a resolved attribute.
#2023-01-2322:34wilkerluciothats odd#2023-01-2322:35Ben GrabowEdited entity -> attr to be more clear#2023-01-2322:35wilkerluciohumm, or maybe something I missed, let me try it here#2023-01-2322:35wilkerlucioah, this is what you can do: (pci/reachable-paths env (pfsd/data->shape-descriptor entity))#2023-01-2322:36wilkerluciowhich is the right way to use reachable-paths, it expects a shape descriptor as input, but I only remembered it now looking it its sources, hehe:
(>defn reachable-paths
  "Discover which paths are available, given an index and a data context.

  Also includes the attributes from available-data."
  [{::keys [index-io] :as env} available-data]
  [(s/keys) ::pfsd/shape-descriptor
   => ::pfsd/shape-descriptor]
#2023-01-2322:39Ben Grabow
(pfsd/data->shape-descriptor {:my-provided-entity {:body "hello world"}})
=>
{:my-provided-entity {:body {}}}
#2023-01-2322:40Ben GrabowIs there a way I can tell this to only parse keys that are in the index? :body is not a pathom attribute, so I think I don't want it mentioned in the query. (It seems to work fine since the parent attr is provided instead of resolved.)#2023-01-2322:41wilkerlucioits fine to mention it in the query because its already present in the data, so Pathom will just get it from the entity, no attempt look it up in the index#2023-01-2322:41wilkerlucioyou could make that filtering, but its fine to keep them there#2023-01-2322:41Ben GrabowI think pfsd/data->shape-descriptor-shallow gets me what I want in this particular case but I'm not sure about downsides in the general case.#2023-01-2322:41wilkerlucioif you don't care about the nested parts, then it should be fine#2023-01-2322:42Ben GrabowThe recursive query handles it šŸ™‚#2023-01-2322:42wilkerlucioyup, makes sense#2023-01-2322:44Ben Grabowv2:
(defn recursive-resolve
  [env entity]
  (let [entity (p.eql/process
                 env
                 entity
                 (pfsd/shape-descriptor->query 
                   (pci/reachable-paths env 
                     (pfsd/data->shape-descriptor-shallow entity))))]
    (->> (for [[k v] entity]
           [k (cond
                (instance? PersistentArrayMap v) (recursive-resolve env v)
                (vector? v) (map #(recursive-resolve env %) v)
                :else v)])
         (into {}))))
#2023-01-2322:45wilkerlucioanother suggestion, use (coll/native-map? v) instead of (instance? PersistentArrayMap v)#2023-01-2322:45Ben GrabowVery nice!#2023-01-2322:45Ben Grabow
(defn recursive-resolve
  [env entity]
  (let [entity (p.eql/process
                 env
                 entity
                 (pfsd/shape-descriptor->query
                   (pci/reachable-paths env
                     (pfsd/data->shape-descriptor-shallow entity))))]
    (->> (for [[k v] entity]
           [k (cond
                (coll/native-map? v) (recursive-resolve env v)
                (vector? v) (map #(recursive-resolve env %) v)
                :else v)])
         (into {}))))
#2023-01-2322:46wilkerlucioalso, since you are reading form vector, mapv instead of map, to keep it as a vector#2023-01-2322:47Ben Grabowyeah lots of rough edges here (including consuming stack!) that would need to be cleaned up before shipping to prod#2023-01-2322:48wilkerlucioone thing though, there is big drawback in mapping over the entities yourself, that is you lose the batch capatibility, since each entity gets its own request#2023-01-2322:48wilkerlucioalso, if you are not using a persistent cache for the planner (you should do it in general), it means its planning again for each instance, also not very efficient#2023-01-2322:51Ben GrabowLuckily I am doing ad-hoc analytical work today so we get to be sloppy about performance šŸ˜†#2023-01-2322:55wilkerluciothe caching is easy and will help a lot, you just need to make something like this:
(def env
  (-> {:com.wsscode.pathom3.connect.planner/plan-cache* (atom {})}
      (pci/register ...)))
#2023-01-2323:02Ben GrabowPlanning and query resolution has been so dang fast every time I've used Pathom that it's never seemed like a priority to me. I have a little over 100 attributes and 100 resolvers in my index. At the REPL queries come back apparently instantly unless my resolvers are doing IO, and in production my typical query takes about 15 ms to plan (which is dwarfed by time spent fetching input data). One day I hope to have a graph big enough and complex enough that I will need the excellent performance optimizations you've made, but I don't need them today!#2023-01-2323:03Ben GrabowI am glad to learn about the plan-cache though, and I will reach for it when I need it!#2023-01-2323:03wilkerluciogreat to hear šŸ™‚#2023-01-2323:03wilkerlucioyeah, take your time, when you do so you will see a drop to 1ms after the first run for planning šŸ™‚#2023-02-1005:59Joel@UANMXF34G trying to use your function above, maybe i just don’t understand how to invoke it.#2023-02-1014:01Ben Grabow@UH13Y2FSA can you show what you have tried?#2023-02-1015:16JoelError printing return value (IllegalArgumentException) at clojure.lang.RT/seqFrom (RT.java:553). Don't know how to create ISeq from: java.lang.Double (recursive-resolve env {:some/id {:body "hi"}}) I’ve used env with p.eql/process and it works. I didn’t follow what :body is.#2023-01-2415:58TommyHow do you guys tend to write your sql queries for pathom resolvers? Something like hugsql makes sense to me, but each query behind a resolver already seems to isolate them pretty well. Honeysql for dynamically creating queries, but I don't ever find that to be necesary (in simple cases), and you also risk it's DSL having a a gap where you have to peer behind the curtain (it does seem that it is fairly comprehensive though). I think im leaning towards hugsql because my data definitions are in .sql files as migrations, so I like the symmetry of queries in .sql files#2023-01-2416:02danierouxHugSQL, and https://pathom3.wsscode.com/docs/resolvers/#batch-resolvers to minimise the queries is working well for us#2023-01-2417:02JYeah same a @U9E8C7QRJ with use batch-resolvers a lot. We have hugsql queries but we want to migrate into honeysql (it’s just data)#2023-01-2417:19Tommywhy do you want to migrate? what benefits would honey have @UHZPYLPU1#2023-01-2417:22Eric DvorsakMy experience was honey then hug then back to honey. I found that hug gets too much in the way when you start to need a bit of parameters, and at this point it doesn't look as neat and having clojure code in you templates is really hard to debug. For most attributes in my current codebase I don't even write any SQL directly and instead use Fulcro rad defattr to define the model, and a custom version of fulcro-rad-sql so the sql writes itself.#2023-01-2507:04J@UBT7FH96Z for me the benefits of honey over hugsql is that honey is just data and you can create directly sql into a .clj. Hugsql doesn’t play well with the repl too.#2023-01-2516:44TommyI do like being able to just (h/values [map]) my data into the database.. @UHZPYLPU1#2023-01-2417:18wilkerlucio#2023-01-2418:34robert-stuttafordhoping a fellow Pathom3 user who's further along can help a noob. i have been butting my head against this for a good few hours. what could i be doing wrong?#2023-01-2418:35robert-stuttafordbtw @U066U8JQJ, if you've advice for how i should involve a Datomic db in a pathom resolver index, i'd love your advice! i can't use the viz tool (i crash it with all the datomic internal data), and any exception i get happens to print the whole Datomic database out, making the feedback loop truly horrible (because Emacs does Not Deal With Big Text Well)#2023-01-2418:36robert-stuttafordI reallllly want to use Pathom3, i want what it offers#2023-01-2418:37caleb.macdonaldblackuser-entity-resolver only has one param when it should probably have two#2023-01-2418:37caleb.macdonaldblackdb will be in first env param and user-id in the second input param#2023-01-2418:38caleb.macdonaldblackResolvers can take a single input arg or two args of env & input#2023-01-2418:39robert-stuttafordso i did try that, with {:db db} as an env and the rest as input, but it didn't seem to make much difference. super happy to give it another shot though.#2023-01-2418:39caleb.macdonaldblackCan you post a snippet?#2023-01-2418:40caleb.macdonaldblack
(pco/defresolver user-entity-resolver
  [{:keys [db]} {:user/keys [id]}]
  {:user/entity (d/entity db id)})
#2023-01-2418:40caleb.macdonaldblackThat’s what that resolver should look like#2023-01-2418:41caleb.macdonaldblackWell it should look like that if you’re query is this: `
[{[:user/id user-id] [:user/entity]}]
#2023-01-2418:42caleb.macdonaldblackHowever if you’re wanting to fetch the user-id from your env, you would do this:
(pco/defresolver user-entity-resolver
  [{:keys [db user-id]} _input]
  {:user/entity (d/entity db user-id)})
#2023-01-2418:42caleb.macdonaldblackand it’s user-id, which is in the env, not user/id#2023-01-2418:45caleb.macdonaldblackAlso in your p.eql/process you’re confusing naming here with pathom conventions. indexes is the env. And that is where you would put the db. where you have env you instead want that to be the entity or input.#2023-01-2418:46caleb.macdonaldblackAnd you can simply put {} if you don’t have any input. You can also omit that entirely and just call it as (p.eql/process indexes eql)#2023-01-2418:47caleb.macdonaldblackYou could also do this and drop the full-name and email resolvers
(pco/defresolver user-entity-resolver
  [{:keys [db user-id]} _input]
  {::pco/output [:user/full-name :user/email]}
  (d/entity db user-id))
#2023-01-2418:48caleb.macdonaldblackAssuming you want the user-id in the env and not the input. Which is safer if you’re allowing eql queries from the client, where the client can put whatever they want in the input.#2023-01-2418:52robert-stuttafordthanks Caleb, i'm going to work through your notes. hopefully all i've done is misunderstand something, and success is close!#2023-01-2418:57caleb.macdonaldblackNo worries. Specifically the misunderstanding is that env/indexes are the same thing. And entity/input is a different thing. Resolvers can take either just the input/entity with one arg [input]. Or both env & input with two args. [env input]#2023-01-2418:57caleb.macdonaldblackEQL queries only have access to data from the input.#2023-01-2418:58caleb.macdonaldblackYou cannot write an eql query that can access data from the env param unless you write a resolver specifically to do this#2023-01-2421:43wilkerlucio@U0509NKGK like @U3XCG2GBZ said, the user-entity-resolver is probably better separating the env and id, glad to assist in case its not working yet#2023-01-2421:44wilkerluciothe db is better on env due to its being a constant across the processing, while :user/id is better as input, since its very context dependent (also allowing things like ident query)#2023-01-2611:59c0rrzinone question about this: if I want to model a domain or set of domains that depend on the context of a user/customer, would it make sense to put them in the env instead of the input? Or is it a better practice to just replicate this parameter as an input in all resolvers?#2023-01-2612:59wilkerlucio@U0UA5TB2L not really, if you think the graph of entities, the relationships of a user/customer will already set the proper boundaries, that should allow for navigation in context, without having to rely on global definitions (like env stuff)#2023-01-2616:36caleb.macdonaldblack@U0UA5TB2L There are a few ways. https://github.com/souenzzo/eql-style-guide/issues/4#issue-651073195#2023-01-2616:40c0rrzinthanks!#2023-01-2422:41Eric DvorsakIf I have 2 entities "user" and "organization" with eg user1 and org1, and I want a resolver for "user/seen-org-prop", the only way to provide the "org" for that resolver in a query is through params, right?#2023-01-2422:42wilkerluciono, you can always use some other input for it#2023-01-2422:42wilkerluciobut maybe I don't fully understand, can you give a concrete example?#2023-01-2422:46Eric Dvorsakyes, do you mean resolvers and queries? or a more precise description? basically I want to return true/false if there is a new data-policy available that the user hasn't seen right now I'm not doing it with a resolver but instead I do a query first then a bit of filtering, this is my query:
[{org-ident [{:organization/latest-data-privacy [:data-privacy/id]}]}
 {user-ident [{:user/data-privacy [:data-privacy/id
                                                        :data-privacy/organization-id
                                                        :data-privacy/version]}]}]
#2023-01-2422:47wilkerluciowhat user/seen-org-prop is about, what does it mean?#2023-01-2422:48Eric Dvorsakto be more correct it would be user/seen-latest-privacy-policy?#2023-01-2422:48Eric Dvorsakwould be true if the data-privacy/id of the user/data-privacy with the highest version for this org matches the one from organization/latest-data-privacy#2023-01-2422:49wilkerluciomaybe a nested input to load the depths from data-privacy and bring it up?#2023-01-2422:51Eric Dvorsakwhat do you mean by load the depths?#2023-01-2422:51wilkerlucioI'm still trying to figure the relationship of your data#2023-01-2422:52wilkerlucioits really about it having a way to trace from the user (maybe from user id) to the final data you need#2023-01-2422:52wilkerluciobut I see your query has an org-ident outside the user-ident boundary, are you trying to relate those separated pieces inside the user-ident?#2023-01-2422:53Eric Dvorsakorganization has a one to many relationship with data-privacy (an org can have multiple data-privacy versions) user has a many to many relationship (a user can have approved data-privacy from multiple orgs and multiple versions)#2023-01-2422:53Eric Dvorsakyes the first subquery with the org ident is using a resolver that returns the most recent data-privacy policy of the org#2023-01-2422:54wilkerlucioyou need to get a way to ask for that from inside the user context, acessing things that are outside the boundary is not really possible#2023-01-2422:54wilkerluciothe user must have a way to link to that order-ident, from inside it#2023-01-2422:56Eric Dvorsak
user-ident [{:user/data-privacy [:data-privacy/id
                                 :data-privacy/organization-id
                                 :organization/latest-data-privacy
                                 :data-privacy/version]}
#2023-01-2422:56Eric Dvorsaki could have it like this#2023-01-2422:56Eric Dvorsakbut then I resolve it for each version of the data policy of each org which I don't care about#2023-01-2423:03Eric DvorsakI guess if I have a user/last-data-privacy resolver as well then it's not a problem anymore#2023-01-2423:04Eric Dvorsakthe last piece is about passing the organization-id, which again I suppose needs to be done via params?#2023-01-2423:05wilkerluciodepends on how you wanna handle it, if the question is based on some user input in the EQL request (like: I want to check this property in relation to this specific org id), them yes, you should use a param#2023-01-2423:06Eric Dvorsakyeah makes sense, user/has-seen-latest-data-policy would return the answer for all the orgs the user is part of unless org-id is provided via params to filter#2023-01-2508:55Eric DvorsakI found two issues related to nested outputs in the regular parser, I'll work on some repro: • if a query asks for nested attributes, the result might return without them (I assume it's because they are already resolved by another but not listed as outputs of the last resolver which drops them), this happens even when lenient mode is false without a warning, if the last resolvers lists those nested attributes as output then they appear in the result • a batch resolver for a nested attribute in a collection only receives one input These issues don't appear with the parallel parser.#2023-01-2508:56Eric DvorsakIn the meatime this is the query I was working with for the second point, where user/latest-data-privacy was returning 2 items, but organization/latest-data-privacy was returning empty for the first, and I could see in the inputs of this resolver that it was only receiving the second one: (pathom/parser {:current-user/id 5} [{[:user/id 16] [{:user/latest-data-privacy [:data-privacy/organization-id {:organization/latest-data-privacy [:data-privacy/id]} :data-privacy/id :data-privacy/version]}]}])#2023-01-2508:59Eric Dvorsakfor the first point I had this query: (pathom/parser {:current-user/id 5} [{[:organization/id 2] [{:organization/latest-data-privacy [:data-privacy/organization-id :data-privacy/id]}]}]) where data-privacy/organization-id will not be in the result organization/latest-data-privacy was not listing data-privacy/organizatio-id as output but the resolver that provided its input did#2023-01-2517:56caleb.macdonaldblackFrom what I can tell. If two resolvers output the same top level key, Pathom3 will treat them as equal regardless of nested params. #2023-01-2517:58caleb.macdonaldblackIn my graph design, I’m particularly careful when modeling relationships. I’ll have resolvers with nested inputs/outputs return an ID attribute only. And the other resolvers that return data for that id. #2023-01-2517:59Eric DvorsakThey don't output the same top-level keys, one resolver gets the inputs that the latest-data-privacy wants as well as some attributes requested by the query, normally they should be in the response but the regular parser drops them. The parallel parser outputs them as expected#2023-01-2518:00Eric DvorsakWhat you suggest to do is producing the case with the bug (the nested output is only the Id)#2023-01-2518:02caleb.macdonaldblackAh ok. Not sure then. Interested to see your example. #2023-01-2518:45Eric DvorsakThis would be the repo for the first point
(ns com.wsscode.pathom3.test
  (:require  [clojure.test :as t]
             [com.wsscode.pathom3.format.shape-descriptor :as pfsd]
             [com.wsscode.pathom3.connect.operation :as pco]
             [com.wsscode.pathom3.connect.indexes :as pci]
             [com.wsscode.pathom3.connect.planner :as pcp]
             [edn-query-language.core :as eql]
             [com.wsscode.pathom3.interface.eql :as p.eql]
             [com.wsscode.pathom3.interface.async.eql :as p.a.eql]))

(pco/defresolver toy
  [env input]
  {::pco/input  [:toy/id]
   ::pco/batch? true
   ::pco/output [:toy/name :toy/id :toy/order]}

  (map #(get {1 {:toy/name "Bobby" :toy/id 1 :toy/order 3}
              2 {:toy/name "Alice" :toy/id 2 :toy/order 2}
              3 {:toy/name "Rene" :toy/id 3 :toy/order 1}}
             (:toy/id %))
       input))

(pco/defresolver child-toys
  [env input]
  {::pco/input  [:child/id]
   ::pco/batch? true
   ::pco/output [{:child/toys [:toy/id]}]}

  [{:child/toys [{:toy/id 1}
                 {:toy/id 2}
                 {:toy/id 3}]}])

(pco/defresolver favorite-toy
  [env input]
  {::pco/input  [{:child/toys [:toy/id :toy/order]}]
   ::pco/batch? true
   ::pco/output [{:child/favorite-toy [:toy/id]}]}

  [{:child/favorite-toy (->> (:child/toys (first input))
                             (sort-by :toy/order)
                             first)}])



(def env (-> (pci/register
              [toy
               child-toys
               favorite-toy])))

(defn test-case []
  (let [query [{[:child/id 1] [{:child/favorite-toy [:toy/name]}]}]]
    (println "Regular")
    (println (p.eql/process env query))
    (println "===========")
    (println "Parallel")
    (println @(p.a.eql/process (assoc env ::p.a.eql/parallel? true) query))))
#2023-01-2518:46Eric Dvorsak
Regular
{[:child/id 1] #:child{:favorite-toy {}}}
===========
Parallel
{[:child/id 1] #:child{:favorite-toy #:toy{:name Rene}}}
This is the output
#2023-01-2518:59Eric Dvorsakhttps://github.com/wilkerlucio/pathom3/issues/177#2023-01-2519:02Eric Dvorsakmight be the same as the issue reported here https://github.com/wilkerlucio/pathom3/issues/173#2023-01-2613:13wilkerluciothanks for the additional repro on that @U03K8V573EC, this is the next thing I'm planning to work on Pathom 3#2023-01-2613:14wilkerlucioand I think they have a good change of being the same issue (#173 & #177)#2023-01-2517:58caleb.macdonaldblackIn my graph design, I’m particularly careful when modeling relationships. I’ll have resolvers with nested inputs/outputs return an ID attribute only. And the other resolvers that return data for that id. #2023-01-2518:59Eric Dvorsakhttps://github.com/wilkerlucio/pathom3/issues/177#2023-01-2521:54Jakub Holý (HolyJak)In P3, how do I make a similar index-explorer defresolver as I did in P2? In P2 I have
(pco/defresolver index-explorer [{::pco/keys [indexes]} _] ; FIXME for P3
  {::pco/input  #{:com.wsscode.pathom.viz.index-explorer/id}
   ::pco/output [:com.wsscode.pathom.viz.index-explorer/index]}
  {:com.wsscode.pathom.viz.index-explorer/index
   (p/transduce-maps
     (remove (comp #{::pc/resolve ::pc/mutate} key))
     indexes)})
I search the docs but found just https://pathom3.wsscode.com/docs/indexes which wasn’t helpful in this regard. Then I found https://pathom3.wsscode.com/docs/debugging/#index-explorer which reads > To learn about this, please https://blog.wsscode.com/pathom/v2/pathom/2.2.0/connect/exploration.html. The behavior of them in Pathom 3 is the same as in Pathom 2. but I am not sure where to find p/transduce-maps or what to replace it with šŸ™
#2023-01-2613:16wilkerluciothe recommend way to make that in Pathom 3 is using the boundary interface: https://pathom3.wsscode.com/docs/eql#boundary-interface#2023-01-2613:18Jakub Holý (HolyJak)I see! And since Fulcro already uses it then I do not need to do anything for index explorer to work šŸŽ‰ Thx a lot!#2023-01-2622:25Jakub Holý (HolyJak)Hm, this does not seem to work. Fulcro uses the boundary interface but my in-browser Fulcro Inspect’s Index Explorer tells me > Seems like the index is not available.#2023-01-2622:25wilkerluciothat's a different problem, Fulcro inspect is not updated for Pathom 3 =/#2023-01-2622:26wilkerlucioto get that working we need to update Pathom Viz in Fulcro Inspect, the catch is that Pathom Viz has migrated to Fulcro 3, while Fulcro Inspect didn't#2023-01-2622:27wilkerlucioso we need to upgrade Fulcro Inspect to Fulcro 3 first, and then update Pathom Viz, or do some weird glue to connect Fulcro 2 and 3 there, but this last one seems a bad option IMO#2023-01-2622:28wilkerluciowhat you can do now is to use Pathom Viz stand alone app, that properly handles Pathom 3, a bit annoying to have one extra thing, but works#2023-01-2622:35Jakub Holý (HolyJak)I see, thank you! Trying it out now.#2023-01-2622:39Jakub Holý (HolyJak)How does Viz see my Fulcro app? Or rather it does not see it, in my case. so how do I connect it? It happily accepted http://localhost:3009/api as the connection url but that doesn’t really seem to be working as no resolvers are shown šŸ™#2023-01-2622:40wilkerlucioit is probably hitting it just fine, but its asking for the Pathom 2 keyword for indexes, and the Pathom 3 doesn't respond to that key#2023-01-2622:40wilkerlucioI guess we may be able to do a dirty fix#2023-01-2622:41wilkerluciobut adding a new resolver ,to expose Pathom 2 key in Pathom 2 index format#2023-01-2622:41Jakub Holý (HolyJak)ah, maybe it does work, I see errors like > ava.lang.RuntimeException: java.lang.Exception: Not supported: class com.fulcrologic.rad.resolvers_common$secure_resolver$fn__39477 > at com.cognitect.transit.impl.WriterFactory$1.write(WriterFactory.java:65)#2023-01-2622:41Jakub Holý (HolyJak)so I guess I need to clean up the resolvers before sending them over, as I remember we needed to do with P2#2023-01-2622:43wilkerluciothere are some key things to do in this scenario: 1. a resolver with the input of com.wsscode.pathom3.connect.indexes/indexes and output of :com.wsscode.pathom.connect/indexes 2. do a similar thing for all the namespaces for the indexes (from ::pci from p3 to ::pc from p2 3. update each resolver and mutation so the keys use pathom 2 names 4. convert input values from EQL to flat sets#2023-01-2622:44wilkerlucioI think I already have this code on Pathom Viz, let me check#2023-01-2622:47wilkerlucioactually here, in the connector sources: https://github.com/wilkerlucio/pathom-viz-connector/blob/master/src/com/wsscode/pathom/viz/ws_connector/pathom3/adapter.cljc#2023-01-2707:10Jakub Holý (HolyJak)Thank you, I will try that. However I suspect I have the same problem in both cases - I need to ā€œcleanā€ the indices to remove stuff that cannot be transferred over transit, namely (resolver) functions. How do you do that? Thats how we fix it for P2: https://github.com/fulcrologic/fulcro-rad-demo/blob/c09d1c30482c63ca2b468c9a7789dd5e80604a50/src/sql/com/example/components/parser.clj#L29-L31#2023-01-2707:16Jakub Holý (HolyJak)For integration approach #2, I have added com.wsscode.pathom.viz.ws-connector.pathom3.adapter/env as the starting env of my parser. I assumed that it exposes indices in fulcro inspect compatible way but it does not seem to be fully the case, as it doesn’t produce the :com.wsscode.pathom.viz.index-explorer/index resolver it is looking for. I see it declares the output of :com.wsscode.pathom.connect/indexes , which is not the same.#2023-01-2709:40Jakub Holý (HolyJak)I need your advice 😭 I am trying to get the external Pathom Viz working and thus to remove from indexes stuff that cannot be transferred over transit. I have fixed that but now P.V. does not like the response 😭 First, my code to ensure the data is transit-able:
(defn protect-attributes-wrapper [mse]
  (fn [env source {:keys [key] :as ast}]
    (if (#{:com.wsscode.pathom.connect/indexes
           ::pci/indexes
           ::pci/index-attributes
           ::pci/index-io
           ::pci/index-resolvers
           ::pci/transient-attrs
           :com.wsscode.pathom3.connect.runner/attribute-errors
           :pathom.viz/support-boundary-interface?}
         key)
      (walk/prewalk (fn [form] (if (fn? form) nil form)) source)
      (mse env source ast))))

; create the plugin
(p.plugin/defplugin clean-indices-plugin {:com.wsscode.pathom3.format.eql/wrap-map-select-entry protect-attributes-wrapper})
and use the plugin with the parser. Now the dev tools in P.V. show: > The result-action mutation handler for mutation com.wsscode.pathom.viz.index-explorer/load-indexes* threw an exception. > #error {:message ā€œResolver com.wsscode.pathom3.connect.indexes->com.wsscode.pathom.connect/index-resolvers--attr-transform exception at path []: No protocol method IOperation.-operation-config defined for type cljs.core/PersistentArrayMap: > {:config {::pco/input [], ::pco/provides {:p/method {}}, ::pco/output [:p/method], ::pco/op-name :p/method-resolver, ::pco/requires {}}, :resolve nil}ā€ So it seems to fail b/c my cleanup code replaced :resolve value, which was a function and thus not transit-able, with nil . So I am in a Catch 22 šŸ˜ž
#2023-01-2710:37wilkerlucio@U0522TWDA to use the Pathom Viz I suggest you use via the pathom viz connector: https://github.com/wilkerlucio/pathom-viz-connector/#connecting-env-pathom-3#2023-01-2710:37wilkerluciousing from it will make sure all the setup/encoding stuff is properly handled#2023-01-2710:38wilkerlucioso you dont have to any transit or anything#2023-01-2710:38wilkerluciobut for your server comms anyway, I suggest you can use transito: https://github.com/wilkerlucio/transito#2023-01-2710:38wilkerluciothis is wrap top of transit, but I handle things like ensure everything encodes, no matter what (so nothing breaks from failed encoding, as functions for example)#2023-01-2710:42Jakub Holý (HolyJak)I do use the connector and p.connector/connect-env . How does it work? Does it start a separate server on a different port, or what?#2023-01-2710:46wilkerlucioyes, it makes a server on your side and connects the app with it#2023-01-2710:46Jakub Holý (HolyJak)When I have added connect-env to my parser’s env and start the standalone Viz, it does not detect anything, so I guess I have to manyally add a connection? http://localhost:<some port> I assume?#2023-01-2710:46wilkerlucioit actually depends, on cljs it will make a connection with the app via web sockets#2023-01-2710:47Jakub Holý (HolyJak)no, this is clj / backend#2023-01-2710:47wilkerluciofor clj it uses http servers (one at the client, one at the server)#2023-01-2710:47wilkerlucioit should detect automatically#2023-01-2710:47wilkerlucioyou just need to load your source after opening the app#2023-01-2710:47wilkerlucioit should add a tab automatically#2023-01-2710:48Jakub Holý (HolyJak)WDYM by ā€œyou just need to load your source after opening the appā€ ? Like load my parser.clj file into the repl?#2023-01-2710:48wilkerlucioyes, run the part that has the connection code (p.connector/connect-env ...)#2023-01-2710:49wilkerlucioone CLJ side I commonly use a pattern on the end of the threading to make the env, eg:
(def env
  (-> {:some "config"}
      (pci/register ...)
      ((requiring-resolve 'com.wsscode.pathom.viz.ws-connector.pathom3/connect-env)
       "debug")))
#2023-01-2710:50wilkerluciothe requiring-resolve here is nice because I can add/remove viz just by changing this line (no need to change anything at the ns declaration)#2023-01-2710:51Jakub Holý (HolyJak)ah, sorry, I did not get that it takes a parser and returns a parser, I though it takes an env and returns an env#2023-01-2710:51wilkerlucioit does take and env and return an env#2023-01-2710:51wilkerluciothat's the env in the example above#2023-01-2710:51Jakub Holý (HolyJak)oh, I am just confused b/c the docstirng on connect-env reads >
Connect a Pathom parser to the Pathom Viz desktop app. The return of this function
> is a new parser,
>
#2023-01-2710:52wilkerlucioare you using the correct namespace?#2023-01-2710:52wilkerluciopathom 3 has a different namespace#2023-01-2710:52wilkerlucio[com.wsscode.pathom.viz.ws-connector.pathom3 :as p.connector]#2023-01-2710:52Jakub Holý (HolyJak)this is com.wsscode.pathom.viz.ws-connector.pathom3/connect-env#2023-01-2710:52wilkerlucioah, you right, that docstring needs changing#2023-01-2710:53wilkerlucioprobably copied over and missed changing it#2023-01-2710:54Jakub Holý (HolyJak)I do not have access to the env b/c I use Fulcro’s P. parser that hides that. Can I use an env plugin? Like this
[(pbip/env-wrap-plugin #(com.wsscode.pathom.viz.ws-connector.pathom3/connect-env % {::pvc/parser-id "jhtest"}))]
but that doesn’t seem to have any effect…
#2023-01-2710:55wilkerluciono, you really need to make that in the env, because it setups up some plugins, it cant be done at an env wrap level#2023-01-2710:57wilkerluciodocs updated, will be right once there is a next release#2023-01-2710:58Jakub Holý (HolyJak)It got connected šŸŽ‰#2023-01-2710:59Jakub Holý (HolyJak)but somehow it does not see my resolvers and attributes, only foreign-indexes-resolver and 3 pathom’s attributes#2023-01-2710:59Jakub Holý (HolyJak)Is order important? I put it into the env before anything else#2023-01-2710:59wilkerlucioyes, order is important#2023-01-2710:59wilkerlucioif you connect before registering, things wont be there#2023-01-2710:59wilkerluciothe connection needs to be last part, after the env is properly ready#2023-01-2711:04Jakub Holý (HolyJak)it works šŸŽ‰ Thx a lot for the advice! I will contribute the necessary changes to Fulcro and document this.#2023-01-2608:35Jakub Holý (HolyJak)Thanks a lot for these, Wilker! > ERROR: ā€œResolver :order/name-resolver returned an invalid response: ā€œsome textā€ I keep forgetting to return a map šŸ˜… and this always saves me šŸ™#2023-01-2617:24kendall.buchananI see there’s a plugin entry point for ::pcr/wrap-resolve which provides access to the input. Is there a way to access the output that’s being used to identify the resolver? Or is there a way to get access to the resolver’s symbol? Basically I’m looking for a way to identify the resolver itself. (This is to tap into our own performance profiling mechanisms.)#2023-01-2617:25wilkerlucioyes, trying to remember from the top of my head, but I think on env you should be able to find a ::pcp/node, which is planning node, that includes the op-name#2023-01-2617:26wilkerlucioalso includes what specific parts are being looked at for that specific call, in the ::pcp/expects key in the node map, but you can also get the full one looking at the index using the op-name#2023-01-2617:28kendall.buchananOkay, that’s wonderful. I think I see it in here.#2023-01-2617:30kendall.buchananYep, that did it. Thank you.#2023-01-2700:51kendall.buchananSecond question of the day! Our index now has a couple hundred resolvers with a few hundred keys. We’re noticing that small changes to queries can have big differences on the performance of the planning step. Are there any principles to keep in mind with respect to improving planning performance?#2023-01-2701:46wilkerlucioa big one is to use a persistent cache for the planner, its just adding an atom to the env at ::pcp/plan-cache* key so the plan cache is preserved between requests#2023-01-2701:47wilkerluciosince the plan does care about the data values (just their shapes instead), the same query (even with different data) will reuse that cache#2023-01-2702:07kendall.buchananYeah, I added that after I sent over the question and it made a radical difference.
#2023-01-2721:00kendall.buchananDoes the plan cache work independently of the values provided by the query’s ident? If, for example, a :user/username is provided, will the plan be cached for all possible values of :user/username?#2023-01-2721:11wilkerlucioit supposed to, but I went to check after you said it here, and I see how it isn't working that way#2023-01-2721:11wilkerluciobut I think there is a simple fix to it, can you please open an issue?#2023-01-2721:15kendall.buchananNo problem, I’ll do that now. Can you imagine any side-effects of using a persistent cache for the planner? I can with the resolver cache, but it seems like the planner cache is more or less a free lunch.#2023-01-2721:19kendall.buchananhttps://github.com/wilkerlucio/pathom3/issues/182#2023-01-2721:19wilkerluciothanks for the issue!#2023-01-2721:20wilkerluciothe major side effect is memory usage that will increase, but the tradeoff weights a lot more on the gains here#2023-01-2721:21wilkerlucioin case the memory usage of the plan cache becomes an issue, then you can switch the basic atom with some more sophisticated caching implementation, an atom is just a type that supports caching, but for pathom what matters is that it implements the cache protocol: https://pathom3.wsscode.com/docs/cache#the-cache-protocol#2023-01-2721:24kendall.buchananGotcha, but with this patch the space taken up in the cache will be considerably less, I imagine.#2023-01-2721:24wilkerlucioyup#2023-01-2721:24wilkerluciofix almost done šŸ™‚#2023-01-2721:24kendall.buchananExcellent, thank you.#2023-01-2721:28wilkerluciofixed on main#2023-01-2721:47kendall.buchananThanks again. I think Pathom is, hands down, one of the best OSS libraries in the Clojure community, by the way.#2023-01-2721:53wilkerluciohumm, Im thinking my solution may have introduced a bug#2023-01-2721:53wilkerluciolet me check here#2023-01-2721:55wilkerlucioyup, bug confirmed, working on it#2023-01-2721:56wilkerlucioI was naive here, because I just changed the cache key, but in the way I did it will always use the same ident value facepalm#2023-01-2721:56wilkerlucioalready have a test exposing it, now time for the fix#2023-01-2722:03wilkerluciocurrent main is working again, but with the ident optimization removed, have to give a bit more thinking to the solution. the issue is that I need to adapt the graph back after, to re-inject the proper ident values there#2023-01-2722:20wilkerluciook, it should be good now on main#2023-01-3016:17kendall.buchananAs a recap, it’s all working as you’d like now?#2023-01-3016:17kendall.buchananIf so, any chance you could cut a new release for Clojars?#2023-01-3016:17kendall.buchanan(There are two patches I’m hoping to apply.)#2023-01-3016:18wilkerlucioyes, it's all running as expected, I plan to cut a release soon, prob today, but max tomorrow#2023-01-3016:18kendall.buchananExcellent, thank you!#2023-01-2710:33Jakub Holý (HolyJak)@wilkerlucio tiny bug at https://github.com/wilkerlucio/pathom3/blob/041daa72b8ced33d2b621ae41371b71ce59bd625/src/main/com/wsscode/pathom3/connect/operation.cljc#L187 - here name is clojure.core/name šŸ˜… and should have been st. like (::op-name config) instead.#2023-01-2714:15wilkerlucioups, at least is a simple one, do you mind opening an issue please?#2023-01-2719:06Jakub Holý (HolyJak)https://github.com/wilkerlucio/pathom3/issues/181#2023-01-2721:48wilkerluciofixed on main#2023-01-2804:57donavanIs there an extension point (or a way of achieving the same result) similar to ::pcr/wrap-resolve in that it runs once per resolver but similar to ::pcr/wrap-merge-attribute in that the result has all its attributes merged in when it’s called?#2023-01-2819:32caleb.macdonaldblackI’d be interested to know more about what you’re trying to do. Do you have an example? If you’re wanting to output arbitrary attributes and feed them into other resolvers, then I’m not sure it could work. Pathom will make a plan using explicitly defined inputs and outputs. It wouldn’t know how to execute the graph if arbitrary attributes are being returned that aren’t specified. #2023-01-3017:05wilkerlucioyeah, would be good to understand what you are trying to do#2023-01-3017:06wilkerluciowrap-resolve and wrap-merge-attribute run in very distinct phases, so you can't have a mapping 1-1 with them, for example, a resolver that returns 5 attributes will have a single call to wrap-resolve, while it will make 5 calls to wrap-merge-attribute#2023-02-1213:33donavanHi @U066U8JQJ and @U3XCG2GBZ; apologies for the late response. I would like the output of a resolver be dependent on it’s inputs. Concretely (and I’m not sure if this is ultimately a good idea or not), I have a resolver that returns the data for a table of items. The items themselves have raw attributes (i.e. from a database entity) and derived attributes such as formatted values that are resolved by other resolvers. Now, I would like to sort the table before returning it and I would like to sort on the formatted values. I would like to do the sorting, in a sense, by operating on the result of the resolver (like ::pcr/wrap-resolve would allow me to do in a plugin or just in the body of the resolver) but I need the other (derived) attributes merged in. Does that make sense?#2023-01-2818:39Jakub Holý (HolyJak)How can P3 defmutations access the env, e.g. to get a db connection? https://pathom3.wsscode.com/docs/mutations/#using-defmutation only demonstrates using input params šŸ™ Reading the spec for the macro it seems it can take 0 - 2 arguments so I guess in the 2 arg version one of them is the env?#2023-01-2818:45wilkerluciosame ways as resolvers, using the 2 args, env is the first#2023-01-2818:49Jakub Holý (HolyJak)Thank you! => https://github.com/wilkerlucio/pathom3-docs/pull/42#2023-01-2819:33wilkerluciothanks, merged!#2023-01-2901:44dvingo+1 I found this super confusing that the order of the parameters is reversed with two args vs one. My default intuition was that the environment would be the first argument for 1-arity as well as 2.. soooo confused at first hah#2023-01-2909:28Jakub Holý (HolyJak)I can see that. Though it makes sense from the UX perspective because you will almost always need the params argument, while env only sometimes (with the exc. of global resolvers and similar). Fortunately the docs are getting ever better šŸ™‚#2023-01-2913:35dvingoI just always provide two arguments now using the _ as needed. One less thing to think about.#2023-01-2916:22souenzzoI think that always require the _env is simpler to understand.#2023-01-3017:02wilkerluciothe 0 - 2 is purely a convenience, its better to think that with 1 the env is being removed, as opposed to think you add env with 2#2023-01-3017:03wilkerluciothe order will not change (for compatibility), pathom 2 also has the same arity order, which makes them compatible#2023-01-3017:04wilkerluciothe reasoning, as @U0522TWDA pointed out, is because its more common to need input without env than the other way around, so pathom provides arity 1 for those cases, there also times you don't care about env or input, then you can go with zero args#2023-01-3017:04wilkerlucionote these suggar features are only available in the defresolver & defmutation macros, using pco/resolver and pco/mutation requires the usage of arity 2 every time#2023-01-3017:04wilkerluciothe macros will generate an arity 2 fn anyway, this is only real signature for resolvers and mutations (its never a multi-arity fn)#2023-01-3017:54dvingoI'm just thinking from API design point of view - reordering parameters is definitely confusing: see slide 34 (https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/32713.pdf)#2023-01-3017:56dvingoespecially b/c they're both hashmaps. But good to know about this background, agreed with Jakub updating the docs will help with this. And good to know I can just use pco/resolver and write my own macro with consistent parameters#2023-01-3018:02wilkerluciore-ordering is a way to see it, if you think multi-arities add to the end, this is contrived but I don't see different arities this way at all, to me each different arity is like having a different function (which is what actually happens, in cljs for example), in that sense, there is no guarantee or expection that must hold between different arities#2023-01-3018:02wilkerlucioone example that we use often is reduce, that in the arity 4 adds a thing at position 3#2023-01-3018:04dvingoyea my mental model is (defresolver [env args]) and (defresolver [env])#2023-01-3010:06Eric DvorsakIs there an easy way to make the parser behave the same as in lenient mode except for a certain kind of exception (in my case authorization exception). Basically I'd like to use the lenient mode for the API but in case a resolver throws an authorization exception it should interrupt and return minimal infos#2023-01-3010:10Eric DvorsakWhat imostly want is to avoid spewing the :com.wsscode.pathom3.entity-tree/entity-tree in the response when the exception is Unauthorized#2023-01-3015:49caleb.macdonaldblackThis is interesting. It would be useful to wire many graphs together with their own configurations. For example running in parallel seems to be all of nothing.#2023-01-3016:41Eric Dvorsakyes the idea is to drop the query entirely in case of unauthorized access to an attribute for now if there's a way to just drop the subquery I'd probably go for it, but one step at a time, and it might not be easy because the authorization attributes are siblings of the others#2023-01-3016:45Eric DvorsakI was initially going for a solution that rewrites resolvers to include authorization attributes in the inputs (and a pending-authorization / authorized resolver for circular dependencies eg child must have parent authorized but you only know the parent after resolving the child) but it turned out to be too slow when dealing with 1000s of attributes whose authorization requires deep recursion. I am now following the path of changing the query which drastically improved the perf in my case https://github.com/yenda/pathauth It would be very close to no-auth perfs once I let parent authorization flow to children#2023-01-3017:59wilkerluciothis goes a bit against the processing model of Pathom, because pathom intentionally doesn't have anything to deal with entities per-see, because for Pathom entities are just generic bags, so from an authorization perspective, just because you can't access attribute :a, that doesn't mean you wont be able to access :b at the same entity, in this thinking, each attribute has potently its own authorization behavior#2023-01-3018:01Eric Dvorsakyes that's a trade-off I'm fine with here. I expect access to unauthorized attributes to be an adversarial use case. No user (UI) would do that#2023-01-3018:02Eric Dvorsakin return I get the advantage that pathom can resolve my authorizations and fail early when its not granted#2023-01-3018:15Eric Dvorsak@U066U8JQJ > this goes a bit against the processing model of Pathom, because pathom intentionally doesn't have anything to deal with entities per-see, because for Pathom entities are just generic bags, so from an authorization perspective, just because you can't access attribute :a, that doesn't mean you wont be able to access :b at the same entity, in this thinking, each attribute has potently its own authorization behavior I agree with this, that's why ideally the behavior I'd be looking for is that when an authorization resolver throws, I have the list of siblings that need to be blocked. But it also means I need to be able to interrupt the subqueries if those siblings are joins. For now it's not too much of a big deal because I'm fine with throwing away the whole query#2023-01-3018:19Eric Dvorsakbut it's too good right now to be able to define authorization like this:
(defattr id :question/id :int
  {...
   pa/auth [:course/authorized?]})
then pathom just adds :course/authorized as a sibling when :question/id shows up in a query and figures out the inputs required to resolve it (currently I do a little more, when an id attribute requires auth, any attribute linked to it does as well)
#2023-01-3019:26Tommyhow do you guys typically structure your crud mutations (more accurately CUD, as read is with resolvers not mutations). I am considering doing just
(org.myproject/crud {:action :{create,delete,update} :type :{user,post,comment} :id 342 :resource {:user/name "tommy" :user/bio "new bio.."}) 
#2023-01-3019:26Tommyi.e. having a single mutation for entire project.#2023-01-3019:28Tommyother option is (org.myproject.user/crud ...) in the same ns as all the other user code. This way I can put code in specific to the user...#2023-01-3019:29Tommythird option is (org.myproject.user/delete ...), (org.myproject.user/create )...#2023-01-3019:31TommyI like first one, because I can make a lot of use of multimethods and specifically keyword https://clojuredocs.org/clojure.core/derive (which seem cool, but I havent used them before). Any thoughts?#2023-01-3020:43wilkerlucioin general I rather have one mutation per operation, not a rule though, but you can also make some helpers that generate resolvers (to avoid the repetition)#2023-01-3116:13wilkerlucio#2023-02-0105:11Tom H.~I’m having trouble finding ~ nvm, I was looking at main :D#2023-01-3116:18caleb.macdonaldblack@wilkerlucio https://github.com/wilkerlucio/pathom3/issues/167 Is this issue related to performance only? Or does it change how things work?#2023-01-3116:19wilkerlucionot even performance really, this one improves the Index Explorer, now attributes that are only nested will also be listed there (they were missing before this issue is fixed)#2023-01-3116:21caleb.macdonaldblackI’m a little confused what the index-explorer is. Is it related to debugging?#2023-01-3116:21wilkerlucioits a tool in Pathom Viz, to explore the available attributes, resolvers and mutations#2023-01-3116:22wilkerlucio;#2023-01-3116:24caleb.macdonaldblackOk so it wouldn’t really change how querying works. Just indexing for debugging?#2023-01-3116:24wilkerlucioyup#2023-01-3116:25wilkerlucioalso for anyone writing tooling based on the attributes index, now there is more info there#2023-01-3116:28caleb.macdonaldblackAh ok interesting. I was playing around with PathomVis yesterday and discoverability wasn’t working as intuitively as I would’ve thought. I figured perhaps this might’ve fix that. It hasn’t though. I show you an example of what I mean#2023-01-3116:37caleb.macdonaldblackI just DM’d you a screen recording.#2023-01-3116:37caleb.macdonaldblackAlso I just realised it’s not even a nested attribute.#2023-01-3116:38caleb.macdonaldblackAnyway thanks for clarifying about that issue#2023-01-3118:17markaddlemanHi @wilkerlucio . In @caleb.macdonaldblack’s recent thread, he brings up ā€œdiscoverabilityā€ which reminded me of a Pathom pain that I’ve wanted to bring up for a while. I have several resolvers that produce the same attribute. Obviously only one of them is executed in any given plan but sometimes it is difficult to know which resolver produced the result. I’d like to propose an enhancement to Pathom Viz that, given an output, it produces a description of which resolvers produced which piece of the result.#2023-01-3118:19markaddlemanI think this feature could simply output map that follows the same structure as the query result where the leaf values are replaced with the name of the resolver.#2023-01-3118:19markaddlemanOn second thought, it might be more complicated than that for collections#2023-01-3118:22nivekuilsounds like a job for a plugin that adds some metadata to the response?#2023-01-3118:23markaddlemanYeah, I’ve been thinking about writing a plugin to handle this#2023-01-3118:23markaddlemanIn every case where I’ve needed it, I’ve just resorted to sticking println statements into the relevant resolvers because it’s easier at the time šŸ˜•#2023-01-3118:57wilkerluciohello Mark!#2023-01-3118:58wilkerluciothere is already something like that, when using the trace tool you should be able to see the execution graph, which will tell you what resolver got called#2023-01-3118:58wilkerluciothis data is also available in the entity itself, if you check the meta on the entity, it includes the whole execution graph#2023-01-3118:59wilkerluciowith a bit of fiddling you can write some helper that given an entity and an attribute, it tells you what resolver (or resolvers, it might have attempted more than one) computed that property#2023-01-3118:59markaddlemanI think I’ve tried the trace tool (it’s been a while since I had this problem). I think it was cumbersome for large plans or output.#2023-01-3118:59markaddlemanI’m sorry I can’t be more specific. Next time I run into this problem, I’ll be sure to report more details#2023-01-3119:00markaddlemanYour suggestion about using metadata and a helper tool is a good idea#2023-01-3119:00markaddlemanI’m glad the data is already there šŸ™‚#2023-01-3119:00markaddlemanThanks!#2023-02-0104:29pfeodrippeYeah, it's so awesome that Wilker exposes all of this information! I am working on a Clerk viewer for the Pathom resolver dependencies where I leverage this. Check https://gist.github.com/pfeodrippe/e61eaaed6adcd87977fe2bb8d435e129, the code is based on one of https://pathom3.wsscode.com/docs/tutorials/babashka (just modified to introduce clerk). The dependency dev-tooling is just a big bag of code where I experiment with lots of stuff, feel free to use anything from there if you want. tool.pathom/process-query should be a drop-in for p/process (arity 3), so you can visualize and inspect any query you already have available on your own code. It's still an experiment, so use it with a grain of salt. I've made a small video (no audio) so you can check this out without having to run it on your own, see the usages in the REPL and later with Clerk.#2023-02-0108:09jeroenvandijkI think Clerk is a nice way to play around with Pathom and to demonstrate it’s usefulness. I was also playing around with it to explore a SQL db (still WIP)#2023-02-0112:26jeroenvandijk@U5R6XUARE Did you consider pushing your work to Clerk garden? I just pushed a https://github.clerk.garden/jeroenvandijk/hello-garden, I hope to extend it with more Pathom later. It is easy to setup and I believe it is doesn’t have many restrictions in what you can run in it#2023-02-0113:46sheluchin@U5R6XUARE Very cool, thank you. I wonder how hard it would be to include a Clerk viewer for graph visualizations like in the Pathom docs. Clerk could be a nice way for sharing repros for troubleshooting and documenting Pathom stuff.#2023-02-0113:48jeroenvandijk@UPWHQK562 I think this discussion is related https://clojurians.slack.com/archives/C035GRLJEP8/p1675146154406729 I’m guessing this will be feasible at some point#2023-02-0113:49jeroenvandijkAlthough this was about README’s for normal html documentation it would already be possible I guess#2023-02-0118:28pfeodrippe@U0FT7SRLP @UPWHQK562 Yeah, we can try doing something, Pathom Viz should already have all of the introspection we need, it’s just a matter of creating a more focused view for any specific needs we have. We can continue the convo in another thread if you want so we don’t add more pings to Wilker’s notification ahahahah #2023-02-0118:32wilkerlucioalso, for some basic stuff, you can use the pathom viz embed, its super easy to setup, a simple iframe does the job#2023-02-0118:33wilkerluciobut, there is a caveat, the simpler usage involves sending data via query params, which has a limit on the payload size, that can overflow with ease by pathom#2023-02-0118:34wilkerlucioso a proper setup involves loading the iframe, and communicating with it via postMessage, that gets rid of the payload size limit, but its not as easy to setup#2023-02-0106:43HukkaI haven't followed https://github.com/wilkerlucio/pathom3/projects/1 to see how quickly it gets new things, but after this week release there doesn't seem to be much to do for Pathom3 anymore. Is it getting close to ready?#2023-02-0111:38wilkerluciohello Tomi! yes, we are getting there! the API has been stable for a while, although Im gonna have a few breaking changes in the next release (on the placeholders entity data feature, see: https://github.com/wilkerlucio/pathom3/issues/187), the good news is that I'm doing it now to prepare to leave the alpha state. fully ready is probably never going to be. one feature that will prob need more development is the foreign parsers, they have their complexities and not many people AFAIK are trying it, so there is likely to be bugs there. but on the rest things seem to be running smoothly#2023-02-0111:42HukkaThat one already has a PR too; are there other looming breaking changes?#2023-02-0111:44wilkerluciothis one was merged yesterday, but pretty minor, only if you are using the ::p.error/missing-output error cause for some special treatment you will be affected: https://github.com/wilkerlucio/pathom3/issues/149#2023-02-0113:26jeroenvandijkFYI, people that use Pathom with core.cache might want to look into https://ask.clojure.org/index.php/12567/multi-threaded-cache-stampede-in-core-cache, I see the https://pathom3.wsscode.com/docs/cache/#using-corecache would be vulnerable to the same issue (cache stampede)#2023-02-0113:28jeroenvandijkI believe the fix is not hard, but I cannot fully confirm it as I haven’t used the core.cache.wrapped myself yet. Conceptually I’m pretty sure it is not hard to work around it though#2023-02-0113:28wilkerluciothat is great to know!#2023-02-0113:31jeroenvandijkThis is how I would solve it (and how I have been working without the wrapped namespace) https://ask.clojure.org/index.php/12567/multi-threaded-cache-stampede-in-core-cache?show=12570#a12570#2023-02-0122:22Jakub Holý (HolyJak)Hello! How do I prevent exceptions in processor responses, as in > {my-mutation {:com.wsscode.pathom3.connect.runner/mutation-error #error { > :cause ā€œMutation my-mutation not foundā€ ..}} ? This is a problem b/c Transit screams when it tries to serialize an Exception. I have https://github.com/fulcrologic/fulcro-rad/blob/ba17f5d3057a18e4af77294fc59b9408451896cd/src/main/com/fulcrologic/rad/pathom3.clj#L82-L93 ::pcr/wrap-mutate that should catch an exception and return data instead but obviously it doesn’t work. Thank you for any tips!#2023-02-0122:24wilkerlucioyou are looking for ::pcr/wrap-mutation-error#2023-02-0122:25wilkerluciohumm, I'm looking close, maybe not, let me try something here#2023-02-0122:34wilkerluciook, your code is actually correct, but I found that the not found is an edge case here#2023-02-0122:34wilkerluciobecause that doesn't go though the wrap-mutate path#2023-02-0122:34wilkerluciobut it can, I got a fix, gonna make some tests and get it out#2023-02-0122:38wilkerluciomy example code:
(p.eql/process
  (-> {:com.wsscode.pathom3.error/lenient-mode? true}
      (pci/register
        [(pco/mutation 'foo2
           {}
           (fn [env params]
             (throw (ex-info "Foi" {}))))])
      (p.plugin/register
        {::p.plugin/id 'err
         ::pcr/wrap-mutate
         (fn [mutate]
           (fn [env params]
             (try
               (mutate env params)
               (catch Throwable err
                 {::pcr/mutation-error (ex-message err)}))))}))
  ['(foo {})
   '(foo2 {})])
; => {foo {:com.wsscode.pathom3.connect.runner/mutation-error "Mutation foo not found"},
; foo2 {:com.wsscode.pathom3.connect.runner/mutation-error "Foi"}}
#2023-02-0122:48wilkerlucio~fixed in main~ wait, not merged yet#2023-02-0122:52wilkerlucionow merged, fixed by https://github.com/wilkerlucio/pathom3/pull/191#2023-02-0216:41Jakub Holý (HolyJak)That was a lightning fast fix, thank you!#2023-02-0205:32Tom H.How can I write a resolver that puts a new key into a collection returned by another resolver? This is what I’m trying to get working:
(pco/defresolver new-key-resolver [{:keys [outside coll]}]
  {::pco/input [:outside {:coll [:inside]}]
   ::pco/output [{:coll [:new-key]}]}
  {:coll (map (fn [{:keys [inside]}]
                {:new-key (+ inside outside)})
           coll)})
#2023-02-0205:32Tom H.But when I try this:
(p.eql/process (pci/register [new-key-resolver])
    {:outside 2
     :coll [{:inside 1}
            {:inside 2}
            {:inside 3}]}
    [{:coll [:new-key]}])
I get
Execution error (ExceptionInfo) at com.wsscode.pathom3.connect.planner/verify-plan!* (planner.cljc:1688).
Pathom can't find a path for the following elements in the query: [:new-key] at path [:coll 0]
#2023-02-0205:33Tom H.But calling the resolver as a function works:
(new-key-resolver
    {:outside 2
     :coll [{:inside 2}
            {:inside 3}
            {:inside 4}]})

; => {:coll ({:new-key 4} {:new-key 5} {:new-key 6})}
#2023-02-0205:33caleb.macdonaldblackYou need to feed :outside into the :coll the first time you get the coll#2023-02-0205:33caleb.macdonaldblackfor example a fetch-coll-by-id resolver. Would fetch the coll and insert the :outside key into it right there#2023-02-0205:34Tom H.but :outside is dependent on :coll , sorry forgot to add that#2023-02-0205:34Tom H.:outside in the particular case I’m working on is a minimum value taken from the items in :coll#2023-02-0205:35Tom H.I suppose I could do that math in the resolver that returns :coll in the first place?#2023-02-0205:35caleb.macdonaldblack^#2023-02-0205:35Tom H.but they’re currently decoupled, the :coll resolver just returns ids#2023-02-0205:36caleb.macdonaldblackIt’s a limitation in Pathom3. Nested resolvers can’t reach up or go backwards. You can only pass data down. This unfortunately ends up in coupling#2023-02-0205:36Tom H.Ah gotcha#2023-02-0205:36caleb.macdonaldblackI’m working on something to address this though.#2023-02-0205:37caleb.macdonaldblackhttps://github.com/CalebMacdonaldBlack/pathom-examples/blob/main/src/main/clj/com/calebmacdonaldblack/cyclone/example/ordering.clj#2023-02-0205:37caleb.macdonaldblackhttps://github.com/CalebMacdonaldBlack/pathom-examples/blob/main/src/main/clj/com/calebmacdonaldblack/cyclone/example2.clj#2023-02-0205:37Tom H.I suppose I could make :coll2 that does the work#2023-02-0205:37Tom H.and leave the decoupled ones alone#2023-02-0205:37caleb.macdonaldblackIt treats nested entities as references and tracks data with those references as a graph executes. Then you can do reverse lookups#2023-02-0205:38caleb.macdonaldblack
(deftest product-with-order-test
  (testing "Product has a relationship with the order it belongs to"
    (let [env (ordering/create-env)]
      (is (= {:com.example.order/products
              [{:com.example.product/title "Green Apple"
                :com.example/order
                {:com.example.order.price/discount 0.1
                 :com.example/id                   "order-1"}}]}
             (p.eql/process
               env
               {:com.example/id                   "order-1"
                :com.example.order.price/discount 0.1
                :com.example.order/products
                [{:com.example/id               "green-apple"
                  :com.example.product/title    "Green Apple"
                  :com.example.product/quantity 4
                  :com.example.product/price    5.0}]}
               [{:com.example.order/products
                 [:com.example.product/title
                  {:com.example/order
                   [:com.example/id
                    :com.example.order.price/discount]}]}]))))))
#2023-02-0205:39caleb.macdonaldblackFor example, this query starts at an order, nests into the products of the order and then nests further (with a reverse-lookup) to access discount#2023-02-0205:40caleb.macdonaldblackAnd the resolvers to make that work look like this:#2023-02-0205:40caleb.macdonaldblack
(pco/defresolver reverse-lookup-for-products-is-an-order
  [_env input]
  {::pco/input  [{:com.example.order/_products [:com.example/id]}]
   ::pco/output [{:com.example/order [:com.example/id]}]}
  (let [{[{:com.example/keys [id]}] :com.example.order/_products} input]
    {:com.example/order {:com.example/id id}}))
#2023-02-0205:41caleb.macdonaldblackI have to use a plugin and generate some resolvers to make this work#2023-02-0205:46Tom H.wow nice, how does it know what to use as the id attribute?#2023-02-0205:47caleb.macdonaldblackAll entities must have a com.example/id attribute in this example#2023-02-0205:47caleb.macdonaldblackFor example, the entity we pass in originally is this:#2023-02-0205:48caleb.macdonaldblack
{:com.example/id                   "order-1"
  :com.example.order.price/discount 0.1
  :com.example.order/products
  [{:com.example/id               "green-apple"
    :com.example.product/title    "Green Apple"
    :com.example.product/quantity 4
    :com.example.product/price    5.0}]}
#2023-02-0205:49caleb.macdonaldblackThis is the magic plugin that makes it work:#2023-02-0205:49caleb.macdonaldblack
(p.plugin/defplugin transact-entity
  {::pcr/wrap-merge-attribute
   (fn [original]
     (fn [{:com.example.db/keys [schema] :as env} {:com.example/keys [id] :as out} k v]
       (let [env (update env :com.example/db
                   (fn [db]
                     (d/db-with (or db (d/empty-db schema))
                                [{:com.example/id id k v}])))]
         (original env out k v))))
   ::pcr/wrap-root-run-graph!
   (fn [original]
     (fn [{:com.example.db/keys [schema] :as env} ast-or-graph entity-tree*]
       (let [env (update env :com.example/db
                   (fn [db]
                     (d/db-with (or db (d/empty-db schema))
                                [@entity-tree*])))]
         (original env ast-or-graph entity-tree*))))})
#2023-02-0205:49caleb.macdonaldblackIt manages state with datascript#2023-02-0205:49caleb.macdonaldblackIt’s very experimental atm.#2023-02-0211:55Jakub Holý (HolyJak)Hi! Do you have any tips for authorization of mutations? I was thinking about using a :wrap-mutation plugin. Ideally I would be able to attach some kind of ā€œmetadataā€ to each mutation (e.g. :allowd-roles #{..}) and retrieve that info in the plugin to allow/deny it…#2023-02-0215:56caleb.macdonaldblackTake a look at https://github.com/souenzzo/eql-style-guide/issues/4 regarding auth in pathom. You can add your own custom attributes to the config map for resolvers and mutations and access them in a variety of ways. As far as implementation goes, you could: 1. Use the https://github.com/souenzzo/eql-style-guide/issues/4 feature which works for mutations too. 2. Use a plugin like ::pcr/wrap-mutate 3. Sanitise the EQL query before processing. 4. Filter the index and remove unauthorised resolvers before processing#2023-02-0215:58caleb.macdonaldblackYou also mention ā€œmetadataā€ which I just wanted to clarify would most likely be attributes within the resolver/mutation config map, along-side ::pco/input & ::pco/output#2023-02-0216:34Jakub Holý (HolyJak)Thanks a lot, Caleb!#2023-02-0322:49Tommyhttps://github.com/yenda/pathauth#2023-02-0322:50Tommythis may also be relevant. The person working on this is around here, and its not finished AFAIK but good reference maybe#2023-02-0518:56Jakub Holý (HolyJak)Thank you, Tommy!#2023-02-0420:44Eric DvorsakI'm getting the insufficient data calling resolver error in batch resolvers even though the attribute it's trying to resolve is optional https://github.com/wilkerlucio/pathom3/blob/main/src/main/com/wsscode/pathom3/connect/runner.cljc#L484-490 Is that a bug or am I doing something wrong? Essentially I have a :question/id and :question/type, based on the type I have a resolver that can return :question.typeA/id or :question.typeB/id` or :question.typeC/id Then I have a query that optionally asks for attributes that are specific to each type. The query returns fine but when I have big lists being queried then I have a huge amount of stacktraces printed from the error above#2023-02-0514:56caleb.macdonaldblackI’ve run into some unexpected behaviour around optionality in general. Not with batch resolvers however. Pathom3 seems to treat missing data as an error. Which I think is counter-intuitive in situations where there are multiple paths. For example a resolver might attempt to fetch data from a database, and then fall-back to a default value when the db doesn’t have anything. In this case, it would be expected behaviour for the db resolver to be missing data. I’ve had to implement a plugin that silences missing-data errors. Also, while I don’t use lenient mode, I do debug with it sometimes. And I have seen errors returned from intermediary attributes for paths that were unable to fulfil and request and superseded by another one that could. These errors were also unexpected because I didn’t ask for this intermediary attribute and my request was ultimately fulfilled with the alternative path. Essentially, I would get an error in my result with lenient mode but no errors at all with strict mode.#2023-02-0611:44wilkerluciohello folks, yeah, sometimes it gets tricky, but in the sense of missing is an error, it should only get the error if there is a required thing and it exausted all the options to get it. @U03K8V573EC I need repro so we can investage the case in question, to see if its a pathom bug or some setup issue#2023-02-0611:45wilkerlucio@U3XCG2GBZ if you send situations like these when you encounter, would be good to chase down possible errors in the processing with optionals#2023-02-0612:37Eric Dvorsak@wilkerlucio thanks, I'll work on a repro, I've noticed that there might be a few issues related to nested inputs in the batch resolution. the parallel parser was working in my example but was 20 times slower making it unusable as a workaround#2023-02-0613:05Eric Dvorsak@wilkerlucio here you go https://github.com/wilkerlucio/pathom3/issues/192#2023-02-0613:41wilkerluciothanks! I'll have a look later today#2023-02-0702:47wilkerlucioI did some investigation, the issue only happens in lenient mode, and doesn't need batch at all, I got down to this repro:
(ns repro-192
  (:require [com.wsscode.pathom3.connect.indexes :as pci]
            [com.wsscode.pathom3.connect.operation :as pco]
            [com.wsscode.pathom3.interface.eql :as p.eql]))

(pco/defresolver book [_]
  {::pco/input  [:book/id]
   ::pco/output [:book/name]}
  {})

(pco/defresolver item [_]
  {::pco/input [:item/id]
   ::pco/output [:item/id :book/id]}
  {})

(def env
  (-> {:com.wsscode.pathom3.error/lenient-mode? true}
      (pci/register
        [book
         item])))

(comment
  (p.eql/process env
    {:item/id 1}
    [:item/id
     :book/name]))
this is a removable feature I think, the idea there was to help to find the error first, but clearly it goes sideways with optionals
#2023-02-0702:47wilkerlucioI'm considering just removing it altogether, there are ways to pull out this information from node stats if wanted, so this part of the process can be simplified#2023-02-0702:48wilkerlucioI'll give some though to it and continue tomorrow#2023-02-0716:23caleb.macdonaldblack@wilkerlucio are you referring to ā€œmissing attributeā€ error or lenient mode as the removable feature?#2023-02-0716:26caleb.macdonaldblackI think you mean the missing attribute error and if so, I agree. If an attribute is optional or available from multiple paths, it would be intentional to have a resolver be unable to provide it.#2023-02-0716:49wilkerlucioI'm referring specifically to the error Insufficient data calling resolver#2023-02-0716:49wilkerluciolenient mode is an important feature, no intention to remove it whatsoever šŸ™‚#2023-02-0716:53wilkerlucioand to be clear, the Insufficient data calling resolver is a debug optimization, to help point where in the chain the output was missed, but other than that it doesn't really affects how the runner works#2023-02-0719:43Eric Dvorsakremoving sounds fine to me my main issue was that I have a wrap-on-resolver-error that logs errors, and usually those are important but here its just logging infinite amounts of things that are not errors additionally there isn't enough information in the error Insufficient data calling resolver to filter it, might be worth adding to ex-data so one can filter without diving into the ex-message and string processing. Eg resolver-name and missing-attribute should def be in there#2023-02-0719:44wilkerlucioI'm very much leaning on removing it, in hindsight the implementation of the idea is bad, I think the optional case demonstrates it clearly#2023-02-0719:48Eric Dvorsakworks for me, then I can remove the code I have to mute the errors in that plugin#2023-02-0720:48caleb.macdonaldblack^ I also muted this error#2023-02-0721:05wilkerluciofixed on main#2023-02-0721:10wilkerlucioplease let me know if that fixed your cases#2023-02-0808:04Eric Dvorsak@wilkerlucio I still get a lot of {:response "Insufficient data calling resolver 'question.drag-order/answers-resolver. Missing attrs :question.drag-order/id"}#2023-02-0808:07Eric Dvorsakmy bad I forgot that I'm overwritting deps locally, after pulling latest pathom3 main no more errors#2023-02-0613:05Eric Dvorsak@wilkerlucio here you go https://github.com/wilkerlucio/pathom3/issues/192#2023-02-0816:41Eric DvorsakIs there a canonical way to purge the cache of a particular resolver from within another? my use case is that I have a mutation that does a query with the parser from the env, this uses resolver A, but the mutation also does some inserts and the initial query needs to return the result of resolver A, but not the cached one from the inner query, rather the updated one after the mutation#2023-02-0818:59caleb.macdonaldblackCould you please provide some examples of input and output and what you’re expecting?#2023-02-0819:11Eric Dvorsak
(ns com.wsscode.pathom3.test
  (:require  [clojure.test :as t]
             [com.wsscode.pathom3.format.shape-descriptor :as pfsd]
             [com.wsscode.pathom3.connect.operation :as pco]
             [com.wsscode.pathom3.connect.indexes :as pci]
             [com.wsscode.pathom3.connect.planner :as pcp]
             [edn-query-language.core :as eql]
             [com.wsscode.pathom3.interface.eql :as p.eql]
             [com.wsscode.pathom3.interface.async.eql :as p.a.eql]))



(def counter (atom 0))

(pco/defresolver get-count
  [env _]
  {::pco/output [:counter]}
  {:counter @counter})

(pco/defmutation inc-count
  [{:keys [parser] :as env} _]
  (println :counter-before (parser env [:counter]))
  (swap! counter inc)
  {})

(def env (-> (pci/register
              [get-count
               inc-count])
             (assoc :com.wsscode.pathom3.error/lenient-mode? true)))

(def parser
  (let [process (p.eql/boundary-interface env)]
    (fn [env tx]
      (process (assoc env :parser process) {:pathom/ast (eql/query->ast tx)}))))

(defn test-case []
  (let [query [{`(inc-count) [:counter]}]]
    (parser {} query))
  )
#2023-02-0819:11Eric Dvorsak
com.wsscode.pathom3.test> (test-case)
:counter-before {:counter 0}
#:com.wsscode.pathom3.test{inc-count {:counter 0}}
com.wsscode.pathom3.test> @counter
1
com.wsscode.pathom3.test> (parser env [:counter])
{:counter 1}
com.wsscode.pathom3.test> 
#2023-02-0819:12Eric Dvorsakif you remove the query in the mutation and reset the counter:#2023-02-0819:12Eric Dvorsak
com.wsscode.pathom3.test> (test-case)
#:com.wsscode.pathom3.test{inc-count {:counter 1}}
com.wsscode.pathom3.test> @counter
1
#2023-02-0819:24caleb.macdonaldblackafaik. Pathom3 will execute mutations first so their results are available to resolvers after. Calling the println in the mutations does seem to cache the result ::pco/cache? = false is something you can do in the resolver to prevent the cached result#2023-02-0819:24Eric Dvorsakwith pco/cache? false on the counter resolver:
com.wsscode.pathom3.test> (parser env [:counter {`(inc-count) [:counter]}])
:counter-before {:counter 0}
{:counter 1, com.wsscode.pathom3.test/inc-count {:counter 1}}
#2023-02-0819:24Eric Dvorsakbut considering the resolver is expensive and used in other situations, ideally I'm looking for a solution to only reset the cache in that particular mutation#2023-02-0819:26wilkerlucio@U03K8V573EC it’s relatively easy to clean a cache, im afk now to get the exact name, but at runner namespace, there is some resolver-cache keyword, and that lives in the env#2023-02-0819:26wilkerlucioits an atom, you can have a look at it, you will see the keys are tuples, you can dissoc the thing you want removed from the cache#2023-02-0819:27wilkerlucioanother option, that may be a bit cleaner, is to configure that resolver (or even a set of resolvers) to use a separate cache store#2023-02-0819:27wilkerluciodoing so, they will have their own atom, this way you can just clean that cache (reseting the atom to an empty map)#2023-02-0819:29caleb.macdonaldblackIf you don’t run the query in the mutation, pathom will execute the mutation first and the resolvers after and you get the incremented value.#2023-02-0819:54Eric Dvorsak@U3XCG2GBZ this is just a repro case, I need to run the query in the mutation for my use case since I'm using the results in the mutation#2023-02-0819:55Eric Dvorsak@U066U8JQJ having a separate atom that can be reset sounds like a good idea.#2023-02-0819:58caleb.macdonaldblackDepending on what you’re doing, you might need to consider any transactional issues. If two requests are run simultaneously, they could both read a state of 0 and then both increment to 1 and write/return that. When you would probably intend on the result being 2#2023-02-0818:30Eric DvorsakFulcro wraps mutation with a plugin to turn exceptions into data:
(letfn [(wrap-mutate-exceptions [mutate]
          (fn [env ast]
            (try
              (mutate env ast)
              (catch Throwable e
                (log/errorf e "Mutation %s failed." (:key ast))
                ;; FIXME: Need a bit more work on returning errors that are handled globally.
                ;; Probably should just propagate exceptions out, so the client sees a server error
                ;; Pathom 2 compatible message so UI can detect the problem
                {:com.wsscode.pathom.core/errors [{:message (ex-message e)
                                                   :data    (ex-data e)}]}))))]
  (p.plugin/defplugin rewrite-mutation-exceptions {::pcr/wrap-mutate wrap-mutate-exceptions}))
One issue I encountered is if my query asks for attributes for the mutation eg
[{'(my-mutation {:paramA 1 :paramB 2) [:sum]}]
then I don't have the error in the result, instead just an empty map {my-mutation {}} Is the only way to get the error to explicitly ask for the attribute :com.wsscode.pathom.core/errors?
#2023-02-0819:03caleb.macdonaldblack:com.wsscode.pathom3.format.eql/map-select-include might help#2023-02-1116:46Jakub Holý (HolyJak)> I don’t have the error in the result Where, in Fulcro or in the actual response (look into the browser Network tab and parse the transit response). If it is missing from Fulcro but present in the actual response then yes, the problem is that F. strips out everything you do not ask for. At one point RAD app did add it automatically https://github.com/fulcrologic/fulcro-rad/commit/b0b8e288d808372480e0177cc98ffa00e487b273 (but not anymore, likely b/c it now needs to support both Pathom 2 and 3 and the key has changed)#2023-02-1119:58Eric Dvorsak@U3XCG2GBZ solution did the trick, it wasn't even for Fulcro in my case I was doing the pathom3 query in an endpoint for a legacy app#2023-02-0820:31JoelI’m not understanding EQL, I have a couple of resolvers like this:
{::pco/output  [{:x/as [:x/a]}]}

{::pco/input   [:x/a]
 ::pco/output  [:x/bs]}
What query would give me each :x/a with it’s corresponding :x/bs?
#2023-02-0820:33caleb.macdonaldblack
[{:x/as [:x/a :x/bs]}]
#2023-02-0820:33caleb.macdonaldblackthis should work too
[{:x/as [:x/bs]}]
#2023-02-0820:34Joeloh, thanks, that worked.#2023-02-0820:35caleb.macdonaldblack
{:x/as {:x/bs :something}}
{:x/as [{:x/bs :something}]}
You can expect data to come back like that depending on the relationship respectively
#2023-02-0917:09caleb.macdonaldblackNot sure what you mean, sorry. Could you send some examples?#2023-02-1005:08Joelis there a way to not mention :x/as in the above query? I just want :x/a :x/bs , which again i suppose i can just use clojure to ā€œtossā€ the :x/as at root. Maybe that’s more what ā€œsmart mapsā€ are for?#2023-02-0907:55PanelAny example of handwritten dynamic resolver ? The graphql one is a bit much to understand what’s going on. I got some resolvers that return external system query, I want to merge them together to make only one call to the api. #2023-02-1011:12wilkerluciohello, I suggest having a look into the datomic implementation, its simpler than the graphql one#2023-02-1011:13wilkerlucioalso the tests for dynamic resolvers, there you can see some examples#2023-02-1011:13wilkerluciohttps://github.com/wilkerlucio/pathom3-datomic#2023-02-1002:10PanelAnything I could do to make legacy-resolve-transaction ( dynamic res ) to be called once with both inputs instead of twice ?#2023-02-1011:29wilkerluciocan you please try ::pcp/experimental-branch-optimizations true in your env see if there is any difference?#2023-02-1011:59PanelIt does make a huge difference !#2023-02-1012:01PanelYou can see with and without the optimizations#2023-02-1012:02wilkerlucioawesome! please let me know if you find any issues using it#2023-02-1019:31caleb.macdonaldblack@U01SBSXRRH6 What is the shape of taxon-requests here? Why is it doing it twice?#2023-02-1019:43wilkerlucio@U3XCG2GBZ its actually normal on this case, but the experimental optimizations were able to handle, which shows in the image with the green nodes#2023-02-1019:57caleb.macdonaldblack@U066U8JQJ Sorry for the confusion, I don’t fully understand what is happening here and I’m just trying to figure it out. I’m not suggesting anything is wrong. Specifically I’m trying to figure out what has caused Pathom to plan executing taxon-requests twice in the first place. Typically I would think of batch resolvers or optional inputs but it doesn’t appear to be the case here.#2023-02-1019:58wilkerluciobut first time I tried to release this optimization it broke some cases to some users, I fixed it but decided to keep it as an optional experimental feature#2023-02-1019:58wilkerluciowhich you can enable with the flag I mentioned here#2023-02-1019:59wilkerlucioit can be cause due to different attributes coming from the same resolver, this is more aparent when using dynamic resolvers#2023-02-1020:06caleb.macdonaldblackAh ok. Got it now.
[{::pco/op-name 'a
  ::pco/input   [:c]
  ::pco/output  [:a]}
 {::pco/op-name 'b
  ::pco/input   [:c]
  ::pco/output  [:b]}
 {::pco/op-name 'c
  ::pco/input   [:d :e]
  ::pco/output  [:c]}
 {::pco/op-name 'd
  ::pco/output  [:d]}
 {::pco/op-name 'e
  ::pco/output  [:e]}]
:a and :b both require :c . So separately diverge down their own paths and duplicate efforts fetching :c. But because they both require :c we can optimise by merging these resolvers into one. Fetch :c one time and calculate both :a and :b at the same time.
#2023-02-1020:31wilkerlucioyup, and with the plan snapshots you can see the compression of nodes happening#2023-02-1020:55caleb.macdonaldblack@U066U8JQJ I have a pretty crazy graph and all my tests still pass after enabling experimental-branch-optimizations#2023-02-1022:21wilkerlucionice, good to hear, I just build more confidence to enable those by default, your input is good one on the positive for it :)#2023-02-1102:58PanelCan this be optimize ? The call to taxon-requests are merged ( when using ::pcp/experimental-branch-optimizations true ) but if I have an attribute coming from another resolver, in this case from taxon-flora-requests, then it doesn't get merged to be passed to the dynamic resolver (legacy-resolve-transaction)#2023-02-1103:12Panelone way to solve that is to create a merging resolver that take attributes from all request producing revolvers.#2023-02-1112:13PanelActually that's not solving anything, both request producing resolver are called no matter what now, so it's just making the graph pretty.#2023-02-1212:36wilkerluciohello @U01SBSXRRH6, not sure if I get the last case, seems with the merge thing you mention things do clean simpler#2023-02-1213:46PanelYes but the merge resolver will trigger every resolver that it's merging so it's not a great solution.#2023-02-1213:48Panelhttps://clojurians.slack.com/files/U01SBSXRRH6/F04NWAA6LVB/image.png This is the most basic example, even with ::pcp/experimental-branch-optimizations true it will call the dynamic res twice.#2023-02-1213:52wilkerluciothanks, can you make a repro for this case? I have this initial idea in mind, that sibling branches with same run-next might be able to merge, gotta explore a bit#2023-02-1214:35Panel#2023-02-1214:36Panel#2023-02-1116:58Jakub Holý (HolyJak)Hello! What are the best practices from invoking the parser from inside a mutation? Context: I have a safe-delete-entity mutation that first checks if the entity is referenced by any other entities and if it is, does not delete it and instead returns a list of those entities - e.g. [[:product/id 123] [:order/id 456]]. Now I want to make the list display-friendly and thus fetch also a nice label for each entity and return instead something nice and uniform, like [{:ident [:product/id 123] , :label "Product X"},{:ident [:order/id 456], :label "Order ABC"}] where the label comes actually from the :order/name or :product:name attribute. Now the complication is that in the case of order, the name is not stored in the db but computed by Pathom from other order attributes. So inside my mutation I want to do something like:
(->> deps-list ; list of idents
     (mapv (fn [idt] {:ident idt, 
                      :label (cond (order? idt) 
                                   (-> (parser env [{idt [:order/name]}]) first val :order/name) 
...)})))
Is that ok? But now I have circular dependency, as the parser depends on the mutation and the mutation on the parser. Can I somehow stick the parser inside env , or what do people do? Thank you!
#2023-02-1117:28Jakub Holý (HolyJak)Ok, so I solved it differently: the mutations returns {:deps [ident1 ident 2 ...]} and I have for each entity registered resolvers like these two:
(pbir/single-attr-resolver :order/id :ident (partial vector :order/id))
(pbir/alias-resolver :order/name :label)
which seems to work fine enough.
#2023-02-1119:35caleb.macdonaldblack@U0522TWDA Did you end up querying on the result of the mutation instead of querying within it?#2023-02-1119:39Jakub Holý (HolyJak)No, I modified the query Fulcro send for the mutation to include the join so Pathom does all the work for me. Essentially (m/returning (rc/nc [:dependencies/id {:dependencies/dependants [:ident :label]}])) in the mutation def. The key part is {:dependencies/dependants [:ident :label]} where I had before only :dependencies/dependants . The tx sent is thus
[{(me/safe-delete-entity
   {:ident [:product/id 123]})
  [:dependencies/id
   {:dependencies/dependants [:ident :label]} 
   :tempids]}]
#2023-02-1120:08caleb.macdonaldblackNice one. That’s how I would’ve done it.#2023-02-1123:20Jakub Holý (HolyJak)Thank you for the confirmation!#2023-02-1306:38Eric Dvorsakfor future reference I use 4 different solutions depending on when I have the data and when I need it. If I already know what I need to query before the mutation then it can be part of the same query as the mutation. I I only know after the mutation then the mutation can return some identifiers then it can be in the attributes of the mutation in my query Those are the easy ones. Then there's when I need the data IN the mutation. There's a plugin that can resolve mutation inputs but I find it awkward, not sure it works for anything more complex than a few attributes when you pass an ident. finally, and that's what you initially wanted to do, I do a query with the parser in the mutation. For that you simply assoc the parser in the env For instance mine looks like this:
(defstate parser
  :start
  (let [process (p.eql/boundary-interface base-env)]
    (fn [env tx]
      (let [ast (pa/auth-query auth-attributes tx)
            {user-id :user/id}  (get-in env [:ring/request :session])
            user-id (or user-id (get env :current-user/id))
            env      (-> env
                         (assoc
                           :parser process
                           :com.wsscode.pathom3.error/lenient-mode? true
                           ;; legacy param support
                           :current-user/id  user-id
                           :now (timestamp/now)
                           :query-params (rpc/combined-query-params ast))
                         (authorization))
            response (process env {:pathom/ast ast})]
        (if (-> response :com.wsscode.pathom3.error/error-data :unauthorized)
          "Unauthorized"
          response)))))
#2023-02-1308:52Jakub Holý (HolyJak)Thank you!